Memory-based processors

ABSTRACT

A memory chip may include: a plurality of memory banks; a data storage configured to store access information indicative of access operations for one or more segments of the plurality of memory banks; and a refresh controller configured to perform a refresh operation of the one or more segments based, at least in part, on the stored access information.

CROSS REFERENCES TO RELATED APPLICATIONS

This application claims the benefit of priority of U.S. ProvisionalPatent Application No. 62/727,653, filed on Sep. 6, 2018, which isincorporated herein by reference in its entirety.

BACKGROUND Technical Field

The present disclosure relates generally to apparatuses for facilitatingmemory-intensive operations. In particular, the present disclosurerelates to hardware chips that include processing elements coupled todedicated memory banks. The present disclosure also relates toapparatuses for improving power efficiency and speed of memory chips. Inparticular, the present disclosure relates system and methods forimplementing partial refreshes, or even no refreshes, on a memory chip.The present disclosure also relates to selectable-sized memory chips anddual-port capabilities on a memory chip.

Background Information

As processor speeds and memory sizes both continue to increase, asignificant limitation on effective processing speeds is the von Neumannbottleneck. The von Neumann bottleneck results from throughputlimitations resulting from conventional computer architecture. Inparticular, data transfer from memory to the processor is oftenbottlenecked compared to actual computations undertaken by theprocessor. Accordingly, the number of clock cycles to read and writefrom memory increases significantly with memory-intensive processes.These clock cycles result in lower effective processing speeds becausereading and writing from memory consumes clock cycles that cannot beused for performing operations on data. Moreover, the computationalbandwidth of the processor is generally larger than the bandwidth of thebuses that the processor uses to access the memory.

These bottlenecks are particularly pronounced for memory-intensiveprocesses, such as neural network and other machine learning algorithms;database construction, indexing searching, and querying; and other tasksthat include more reading and writing operation than data processingoperations.

Additionally, the rapid growth in volume and granularity of availabledigital data has created opportunities to develop machine learningalgorithms and has enabled new technologies. However, it has alsobrought cumbersome challenges to the world of data bases and parallelcomputing. For example, the rise of social media and the Internet ofThings (IoT) creates digital data at a record rate. This new data can beused to create algorithms for a variety of purposes, ranging from newadvertising techniques to more precise control methods of industrialprocesses. However, the new data has been difficult to store, process,analyze and handle.

New data resources can be massive, sometimes in the order of peta- tozettabytes. Moreover, the growth rate of these data resources may exceeddata processing capabilities. Therefore, data scientists have turned toparallel data processing techniques, to tackle these challenges. In aneffort to increase computation power and handle the massive amount ofdata, scientists have attempted to create systems and methods capable ofparallel intensive computing. But these existing systems and methodshave not kept up with the data processing requirements, often becausethe techniques employed are limited by their demand of additionalresources for data management, integration of segregated data, andanalysis of the sectioned data.

To facilitate the manipulation of large data sets, engineers andscientists now seek to improve the hardware used to analyze data. Forexample, new semiconductor processors or chips (such as those describedherein) may be designed specifically for data intensive tasks byincorporating memory and processing functions in a single substratefabricated in technologies more fitting for memory operations ratherthan arithmetic computation. With integrated circuits specificallydesigned for data-intensive tasks, it is possible to meet the new dataprocessing requirements. Nonetheless, this new approach to tackle dataprocessing of large data sets requires solving new issues in chip designand fabrication. For instance, if the new chips designed for dataintensive tasks are manufactured with fabrication techniques andarchitectures used for common chips, they would have a poor performanceand/or unacceptable yields. In addition, if the new chips are designedto operate with current data handling methods, they will have poorperformance because current methods can limit the chip's ability tohandle parallel operations.

The present disclosure describes solutions for mitigating or overcomingone or more of the problems set forth above, among other problems in theprior art.

SUMMARY

In some embodiments, a memory chip may include: a plurality of memorybanks; a data storage configured to store access information indicativeof access operations for one or more segments of the plurality of memorybanks; and a refresh controller configured to perform a refreshoperation of the one or more segments based, at least in part, on thestored access information.

Some embodiments may include a non-transitory computer-readable mediumstoring instructions that, when executed by at least one processor,cause the at least one processor to: receive higher-level computer code;identify a plurality of memory segments distributed over a plurality ofmemory banks associated with a memory chip to be accessed by thehigher-level computer code; assess the higher-level computer code toidentify a plurality of memory read commands to occur over a pluralityof memory access cycles; cause a distribution of data, associated withthe plurality of memory read commands, across each of the plurality ofmemory segments such that each of the plurality of memory segments isaccessed during each of the plurality of memory access cycles.

Some embodiments may include a non-transitory computer-readable mediumstoring instructions that, when executed by at least one processor,cause the at least one processor to: receive higher-level computer code;identify a plurality of memory segments distributed over a plurality ofmemory banks associated with a memory chip to be accessed by thehigher-level computer code; assess the higher-level computer code toidentify a plurality of memory access commands each implicating one ormore of the plurality of memory segments; based on analysis of thememory access commands and for each memory segment among the pluralityof memory segments, track an amount of time that would accrue from alast access to the memory segment; and in response to a determinationthat an amount of time since a last access for any particular memorysegment would exceed a predetermined threshold, introduce into thehigher-level computer code at least one of a memory refresh command or amemory access command configured to cause an access to the particularmemory segment.

In some embodiments, a memory chip having a software-configurable memoryrefresh control may include: a plurality of memory banks and a pluralityof memory segments included in each of the plurality of memory banks;and a refresh controller including a data storage element storing atleast one memory refresh pattern to be implemented in refreshing theplurality of memory segments included in each of the plurality of memorybanks, and wherein the memory refresh pattern is configurable usingsoftware to identify which of the plurality of memory segments in aparticular memory bank are to be refreshed during a refresh cycle andwhich of the plurality of memory segments in the particular memory bankare not to be refreshed during the refresh cycle.

Consistent with one embodiment of the present disclosure, the method forproducing selectable sized memory chips from a wafer is provided. Themethod includes forming a wafer comprising a group of dies and acoupling circuit, where the dies of the group comprise memory units, andwhere the coupling circuit is coupled to the dies of the group. Themethod further includes forming the selectable sized memory chips by atleast one of cutting a region of the wafer that comprises the group ofdies and the coupling circuit, to provide a single memory chip thatcomprises the group of dies, and configuring at least one controllerassociated with the group of dies, to control the group of dies togetheras a single memory chip, and cutting regions of the wafer, each regionof the wafer including a die of the group to provide a plurality ofseparate memory chips.

Consistent with another embodiment of the present disclosure, a methodfor producing selectable sized memory chips from a wafer is provided.The method includes, forming a plurality of memory chips on the wafer,the plurality of memory chips arranged along one or more rows, andforming at least one shared input-output bus corresponding to the one ormore rows, wherein the at least one shared input-output bus electricallyconnects at least two of the plurality of memory chips to functiontogether as a single chip. The method may further include cutting atleast two of the plurality of memory chips off the wafer with at leastone corresponding portion of the shared input-output bus such that atleast one memory controller is configurable to control the at least twoof the plurality of memory chips to function together as a single chip.

Consistent with another embodiment of the present disclosure, a waferincluding memory chips with selectable sizes, is provided. The wafer mayinclude a plurality of memory chips arranged along one or more rows ofthe wafer and at least one shared input-output bus corresponding to theone or more rows. The wafer may further include a plurality of cuttingshapes including the plurality of memory chips and excluding at least aportion of the shared input-output bus.

Some embodiments may include a memory instance providing dual-portfunctionality. The memory instance may include a plurality of memorybanks arranged along at least one row and at least one column; at leastone column multiplexer configured to receive, during a single clockcycle, two addresses for reading or writing; and a row decoderconfigured to cooperate with the at least one column multiplexer to:activate a word line based on the two addresses, wherein the two addressshare a word line address; retrieve, during a memory access cycle, afirst of the two addresses from the at least one column multiplexer bydecoding a bitline corresponding to the first address, and retrieve,during the memory access cycle, a second of the two addresses from theat least one column multiplexer by decoding a bitline corresponding tothe second address

In other embodiments, a memory instance providing dual-portfunctionality may include: a plurality of memory banks arranged along atleast one row and at least one column; at least one row multiplexer orat least one column multiplexer configured to receive, during a singleclock cycle, two addresses for reading or writing; and a row decoder anda column decoder configured to cooperate to: retrieve, during a firstcycle, a first of the two addresses from the at least one rowmultiplexer or the at least one column multiplexer, and decode a wordline and a bitline corresponding to the first address, and retrieve,during a second cycle that follows the first cycle, a second of the twoaddresses from the at least one row multiplexer or the at least onecolumn multiplexer, and decode a word line and a bitline correspondingto the second address.

In some embodiments, a memory instance may include: a plurality ofmemory banks arranged along at least one row and at least one column; atleast one row circuit and at least one column circuit configured tofunction as switches; and a row decoder and a column decoder configuredto cooperate to: during a memory clock cycle, retrieve a first of twoaddresses by: activating ones of the at least one row circuit and the atleast one column circuit corresponding to the first address bytransmitting one or more control signals to close ones of the switchingelements corresponding to the first address, and using at least one rowmultiplexer and at least one column multiplexer of the memory chip todecode a word line and a bitline corresponding to the first address, andduring the memory clock cycle, retrieve a second of the two addressesby: activating ones of the at least one row circuit and the at least onecolumn circuit corresponding to the second address by transmitting oneor more control signals to close ones of the switching elementscorresponding to the second address, and using the at least one rowmultiplexer and the at least one column multiplexer to decode a wordline and a bitline corresponding to the second address.

In other embodiments, a memory mat for dual-port access may include: aplurality of memory cells arranged along at least one row and at leastone column and comprising a plurality of capacitive elements; at leasttwo row decoders for activating the at least one row; at least twocolumn multiplexers for activating the at least one column; and a firstplurality of conductive lines connecting the memory cells to a first ofthe at least two row decoders and a first of the at least two columnmultiplexers; and a second plurality of conductive lines connecting thememory cells to a second of the at least two row decoders and a secondof the at least two column multiplexers.

In some embodiments, an integrated circuit may comprise a memory unitcomprising memory cells, an output port, and read circuitry; and aprocessing unit. The read circuitry may comprise a reduction unit and afirst group of in-memory read paths for outputting up to a first numberof bits through the output port. The processing unit may be configuredto send to the memory unit a read request for reading a second number ofbits from the memory unit. The reduction unit may be configured tocontrol the in-memory read paths, during a read operation triggered bythe read request, based on the first number of bits and the secondnumber of bits.

Another embodiment may include integrated circuit that comprises amemory unit comprising memory cells, an output port, and writecircuitry; and a processing unit. The write circuitry may comprise areduction unit and a first group of in-memory write paths for outputtingup to a first number of bits through the output port. The processingunit may be configured to send to the memory unit a write request forwriting a second number of bits from the memory unit. The reduction unitmay be configured to control the in-memory write paths, during a writeoperation triggered by the write request, based on the first number ofbits and the second number of bits.

Some embodiments may include a method for energy reduction of anintegrated circuit. The method may include sending, by a processing unitof the integrated circuit and to a memory unit of the integratedcircuit, a read request for reading a second number of bits from thememory unit; wherein the memory unit comprises memory cells, an outputport, and read circuitry that comprises a reduction unit and a firstgroup of in-memory read paths for outputting up to a first number ofbits through the output port; and controlling, by the reduction unit,the in-memory read paths, during a read operation triggered by the readrequest, based on the first number of bits and the second number ofbits.

In some embodiments, a memory unit may include a first memory mat; asecond memory mat; and an activation unit that is configured to activatea first group of memory cells included in the first memory mat withoutactivating a second group of memory cells included in the second memorymat, wherein the first group of memory cells and the second group ofmemory cells both belong to a single row of the memory unit.

Another embodiment may include a memory unit that comprises a firstmemory mat, a second memory mat; and an activation unit that isconfigured to supply an activation signal to a first group of memorycells of the first memory mat and delay a supply of the activationsignal to a second group of memory cells of the second memory mat atleast until activation of the first group of memory cells has beencompleted. The first group of memory cells and the second group ofmemory cells may belong to a single row of the memory unit.

In some embodiments, a memory unit may comprise a first memory mat, asecond memory mat, and an isolation unit. The isolation unit may beconfigured to isolate first memory cells of the first memory mat fromsecond memory cells of the second memory mat during an initialactivation period in which the first memory cells are activated; andcouple the first memory cells to the second memory cells following theinitial activation period; wherein the first and second memory cellsbelong to a single row of the memory unit.

Some embodiments may include a method for operating a memory unit. Themethod may include activating, by an activation unit, a first group ofmemory cells included in a first memory mat of the memory unit withoutactivating a second group of memory cells included in a second memorymat of the memory unit. The first group of memory cells and the secondgroup of memory cells may both belong to a single row of the memoryunit.

In some embodiments, an integrated circuit may comprise a substrate; amemory array disposed on the substrate; a processing array disposed onthe substrate; and an interface disposed on the substrate. The memoryarray may comprise multiple memory banks, the processing array maycomprise a plurality of testing units, the plurality of testing unitsmay be configured to test the multiple memory banks to provide testresults, and the interface may be configured to output, outside theintegrated circuit, information indicative of the test results.

Another embodiment may include an integrated circuit comprising asubstrate; a memory array disposed on the substrate; a processing arraydisposed on the substrate; and an interface disposed on the substrate.The memory array may comprise multiple memory banks, the interface maybe configured to receive configuration information that comprisesinstructions, the processing array may be configured to execute theinstructions to access the memory array, perform computationaloperations and provide results, and the interface may be configured tooutput, outside the integrated circuit, information indicative of theresults.

Some embodiments may include a method for testing memory banks of anintegrated circuit. The method may include receiving a request to testmemory banks of an integrated circuit, the integrated circuit comprisinga substrate, a memory array that is disposed on the substrate andcomprises the memory banks, a processing array disposed on thesubstrate, and an interface disposed on the substrate; wherein theprocessing array comprises a plurality of testing units; testing, by theplurality of testing units and in response to the request, the multiplememory banks to provide test results; and outputting, by the interfaceand outside the integrated circuit, information indicative of the testresults.

In some embodiments, a method for testing memory banks of an integratedcircuit may comprise receiving by an interface of an integrated circuit,configuration information that comprises instructions, the integratedcircuit comprising a substrate, a memory array that comprises memorybank and is disposed on the substrate, a processing array disposed onthe substrate; and an interface disposed on the substrate; executing, bythe processing array, the instructions by accessing the memory array,performing computational operations and provide result; and outputting,by the interface and outside the integrated circuit, informationindicative of the results.

Consistent with other disclosed embodiments, non-transitorycomputer-readable storage media may store program instructions, whichare executed by at least one processing device and perform any of themethods described herein.

The foregoing general description and the following detailed descriptionare exemplary and explanatory only and are not restrictive of theclaims.

BRIEF DESCRIPTION OF THE DRAWINGS

The accompanying drawings, which are incorporated in and constitute apart of this disclosure, illustrate various disclosed embodiments. Inthe drawings:

FIG. 1 is a diagrammatic representation of a central processing unit(CPU).

FIG. 2 is a diagrammatic representation of a graphics processing unit(GPU).

FIG. 3A is a diagrammatic representation of an embodiment of anexemplary hardware chip consistent with the disclosed embodiments.

FIG. 3B is a diagrammatic representation of another embodiment of anexemplary hardware chip consistent with the disclosed embodiments.

FIG. 4 is a diagrammatic representation of a generic command executed byan exemplary hardware chip consistent with the disclosed embodiments.

FIG. 5 is a diagrammatic representation of a specialized commandexecuted by an exemplary hardware chip consistent with the disclosedembodiments.

FIG. 6 is a diagrammatic representation of a processing group for use inan exemplary hardware chip consistent with the disclosed embodiments.

FIG. 7A is a diagrammatic representation of a rectangular array ofprocessing groups consistent with the disclosed embodiments.

FIG. 7B is a diagrammatic representation of an elliptical array ofprocessing groups consistent with the disclosed embodiments.

FIG. 7C is a diagrammatic representation an array of hardware chipsconsistent with the disclosed embodiments.

FIG. 7D is a diagrammatic representation another array of hardware chipsconsistent with the disclosed embodiments.

FIG. 8 is a flowchart depicting an exemplary method for compiling aseries of instructions for execution on an exemplary hardware chipconsistent with the disclosed embodiments.

FIG. 9 is a diagrammatic representation of a memory bank.

FIG. 10 is a diagrammatic representation of a memory bank.

FIG. 11 is a diagrammatic representation of an embodiment of anexemplary memory bank with sub-bank controls consistent with thedisclosed embodiments.

FIG. 12 is a diagrammatic representation of another embodiment of anexemplary memory bank with sub-bank controls consistent with thedisclosed embodiments.

FIG. 13 is a block diagram of an exemplary memory chip, consistent withdisclosed embodiments.

FIG. 14 is a block diagram of an exemplary redundant logic block set,consistent with disclosed embodiments.

FIG. 15 is a block diagram for an exemplary logic block, consistent withdisclosed embodiments.

FIG. 16 are block diagrams of exemplary logic blocks connected with abus, consistent with disclosed embodiments.

FIG. 17 is a block diagram for exemplary logic blocks connected inseries, consistent with disclosed embodiments.

FIG. 18 is a block diagram of exemplary logic blocks connected in atwo-dimension array, consistent with disclosed embodiments.

FIG. 19 is a block diagram for exemplary logic blocks in a complexconnection, consistent with disclosed embodiments.

FIG. 20 is an exemplary flow chart illustrating a redundant blockenabling process, consistent with disclosed embodiments.

FIG. 21 is an exemplary flow chart illustrating an address assignmentprocess, consistent with disclosed embodiments.

FIG. 22 provides block diagrams for exemplary processing devices,consistent with disclosed embodiments.

FIG. 23 is a block diagram of an exemplary processing device, consistentwith disclosed embodiments.

FIG. 24 includes exemplary memory configuration diagrams, consistentwith disclosed embodiments.

FIG. 25 is an exemplary flowchart illustrating a memory configurationprocess, consistent with disclosed embodiments.

FIG. 26 is an exemplary flowchart illustrating a memory read process,consistent with disclosed embodiments.

FIG. 27 is an exemplary flowchart illustrating a process execution,consistent with disclosed embodiments.

FIG. 28 shows an example memory chip with a refresh controller,consistent with the present disclosure.

FIG. 29A shows an example refresh controller consistent with the presentdisclosure.

FIG. 29B shows another example refresh controller consistent with thepresent disclosure.

FIG. 30 is an example flowchart of a process executing by a refreshcontroller consistent with the present disclosure.

FIG. 31 is an example flowchart of a process implemented by a compilerconsistent with the present disclosure.

FIG. 32 is another example flowchart of a process implemented by acompiler consistent with the present disclosure.

FIG. 33 shows an example refresh controller configured by storedpatterns consistent with the present disclosure.

FIG. 34 is an example flowchart of a process implemented by softwarewithin a refresh controller consistent with the present disclosure.

FIG. 35A shows an example wafer, including dies consistent with thepresent disclosure.

FIG. 35B shows an example memory chip connected to an input/output busconsistent with the present disclosure.

FIG. 35C shows an example wafer including memory chips arranged in rows,and connected to input-output buses consistent with the presentdisclosure.

FIG. 35D shows two memory chips forming a group, and connected toinput-output buses consistent with the present disclosure.

FIG. 35E shows an example wafer, including dies placed in a hexagonallattice and connected to input-output buses consistent with the presentdisclosure.

FIGS. 36A-36D show various possible configurations of memory chipsconnected to an input/output bus consistent with the present disclosure.

FIG. 37 shows an example grouping of dies sharing a glue logicconsistent with the present disclosure.

FIGS. 38A-38B show example cuts through a wafer consistent with thepresent disclosure.

FIG. 38C shows an example arrangement of dies on a wafer and arrangementof input-output buses consistent with the present disclosure.

FIG. 39 shows example memory chips on a wafer with interconnectedprocessor subunits consistent with the present disclosure.

FIG. 40 is an example flowchart of a process of laying out a group ofmemory chips from a wafer consistent with the present disclosure.

FIG. 41A is another example flowchart of a process of laying out a groupof memory chips from a wafer consistent with the present disclosure.

FIGS. 41B-41C are example flowcharts of processes of determining cuttingpatterns for cutting one or more groups of memory chips from a waferconsistent with the present disclosure.

FIG. 42 shows an example of circuitry within a memory chip providingdual-port access along columns, consistent with the present disclosure.

FIG. 43 shows an example of circuitry within a memory chip providingdual-port access along rows, consistent with the present disclosure.

FIG. 44 shows an example of circuitry within a memory chip providingdual-port access along both rows and columns, consistent with thepresent disclosure.

FIG. 45A shows a dual-read using duplicated memory arrays or mats.

FIG. 45B shows a dual-write using duplicated memory arrays or mats.

FIG. 46 shows an example of circuitry within a memory chip withswitching elements for dual-port access along rows, consistent with thepresent disclosure.

FIG. 47A is an example flowchart of a process for providing dual-portaccess on a single-port memory array or mat, consistent with the presentdisclosure.

FIG. 47B is an example flowchart of another process for providingdual-port access on a single-port memory array or mat, consistent withthe present disclosure.

FIG. 48 shows another example of circuitry within a memory chip memorychip providing dual-port access along both rows and columns, consistentwith the present disclosure.

FIG. 49 shows an example of switching elements for dual-port accesswithin a memory mat, consistent with the present disclosure.

FIG. 50 illustrates an example integrated circuit with a reduction unitconfigured to access partial words, consistent with the presentdisclosure.

FIG. 51 illustrates a memory bank for using a reduction unit asdescribed for FIG. 50.

FIG. 52 illustrates a memory bank using a reduction unit integrated intoPIM logic, consistent with the present disclosure.

FIG. 53 illustrates a memory bank using a PIM logic to activate switchesfor accessing partial words, consistent with the present disclosure.

FIG. 54A illustrates a memory bank with segmented column multiplexes fordeactivating to access partial words, consistent with the presentdisclosure.

FIG. 54B is an example flowchart of a process for partial word access ina memory, consistent with the present disclosure.

FIG. 55 illustrates an existing memory chip that includes multiplememory mats.

FIG. 56 illustrates an example memory chip with activation circuits forreducing power consumption during opening of a line, consistent with thepresent disclosure.

FIG. 57 illustrates another example memory chip with activation circuitsfor reducing power consumption during opening of a line, consistent withthe present disclosure.

FIG. 58 illustrates yet another example memory chip with activationcircuits for reducing power consumption during opening of a line,consistent with the present disclosure.

FIG. 59 illustrates an additional example memory chip with activationcircuits for reducing power consumption during opening of a line,consistent with the present disclosure.

FIG. 60 illustrates an example memory chip with a global word line andlocal word lines for reducing power consumption during opening of aline, consistent with the present disclosure.

FIG. 61 illustrates another example memory chip with a global word lineand local word lines for reducing power consumption during opening of aline, consistent with the present disclosure.

FIG. 62 is an example flowchart of a process for sequential opening oflines in a memory, consistent with the present disclosure.

FIG. 63 illustrates an existing tester for memory chips.

FIG. 64 illustrates another existing tester for memory chips.

FIG. 65 illustrates an example of testing memory chips using logic unitson the same substrate as the memory, consistent with the presentdisclosure.

FIG. 66 illustrates another example of testing memory chips using logicunits on the same substrate as the memory, consistent with the presentdisclosure.

FIG. 67 illustrates yet another example of testing memory chips usinglogic units on the same substrate as the memory, consistent with thepresent disclosure.

FIG. 68 illustrates an additional example of testing memory chips usinglogic units on the same substrate as the memory, consistent with thepresent disclosure.

FIG. 69 illustrates a further example of testing memory chips usinglogic units on the same substrate as the memory, consistent with thepresent disclosure.

FIG. 70 is an example flowchart of a process for testing memory chips,consistent with the present disclosure.

FIG. 71 is an example flowchart of another process for testing memorychips, consistent with the present disclosure.

DETAILED DESCRIPTION

The following detailed description refers to the accompanying drawings.Wherever convenient, the same reference numbers are used in the drawingsand the following description to refer to the same or similar parts.While several illustrative embodiments are described herein,modifications, adaptations and other implementations are possible. Forexample, substitutions, additions or modifications may be made to thecomponents illustrated in the drawings, and the illustrative methodsdescribed herein may be modified by substituting, reordering, removing,or adding steps to the disclosed methods. Accordingly, the followingdetailed description is not limited to the disclosed embodiments andexamples. Instead, the proper scope is defined by the appended claims.

Processor Architecture

As used throughout this disclosure, the term “hardware chip” refers to asemiconductor wafer (such as silicon or the like) on which one or morecircuit elements (such as transistors, capacitors, resistors, and/or thelike) are formed. The circuit elements may form processing elements ormemory elements. A “processing element” refers to one or more circuitelements that, together, perform at least one logic function (such as anarithmetic function, a logic gate, other Boolean operations, or thelike). A processing element may be a general-purpose processing element(such as a configurable plurality of transistors) or a special-purposeprocessing element (such as a particular logic gate or a plurality ofcircuit elements designed to perform a particular logic function). A“memory element” refers to one or more circuit elements that can be usedto store data. A “memory element” may also be referred to as a “memorycell.” A memory element may be dynamic (such that electrical refreshesare required to maintain the data store), static (such that datapersists for at least some time after power loss), or non-volatilememories.

Processing elements may be joined to form processor subunits. A“processor subunit” may thus comprise a smallest grouping of processingelements that may execute at least one task or instructions (e.g., of aprocessor instruction set). For example, a subunit may comprise one ormore general-purpose processing elements configured to executeinstructions together, one or more general-purpose processing elementspaired with one or more special-purpose processing elements configuredto execute instructions in a complementary fashion, or the like. Theprocessor subunits may be arranged on a substrate (e.g., a wafer) in anarray. Although the “array” may comprise a rectangular shape, anyarrangement of the subunits in the array may be formed on the substrate.

Memory elements may be joined to form memory banks. For example, amemory bank may comprise one or more lines of memory elements linkedalong at least one wire (or other conductive connection). Furthermore,the memory elements may be linked along at least one addition wire inanother direction. For example, the memory elements may be arrangedalong wordlines and bitlines, as explained below. Although the memorybank may comprise lines, any arrangement of the elements in the bank maybe used to form the bank on the substrate. Moreover, one or more banksmay be electrically joined to at least one memory controller to form amemory array. Although the memory array may comprise a rectangulararrangement of the banks, any arrangement of the banks in the array maybe formed on the substrate.

As further used throughout this disclose, a “bus” refers to anycommunicative connection between elements of a substrate. For example, awire or a line (forming an electrical connection), an optical fiber(forming an optical connection), or any other connection conductingcommunications between components may be referred to as a “bus.”

Conventional processors pair general-purpose logic circuits with sharedmemories. The shared memories may store both instruction sets forexecution by the logic circuits as well as data used for and resultingfrom execution of the instruction sets. As described below, someconventional processors use a caching system to reduce delays inperforming pulls from the shared memory; however, conventional cachingsystems remain shared. Conventional processors include centralprocessing units (CPUs), graphics processing units (GPUs), variousapplication-specific integrated circuits (ASICs), or the like. FIG. 1shows an example of a CPU, and FIG. 2 shows an example of a GPU.

As shown in FIG. 1, a CPU 100 may comprise a processing unit 110 thatincludes one or more processor subunits, such as processor subunit 120 aand processor subunit 120 b. Although not depicted in FIG. 1, eachprocessor subunit may comprise a plurality of processing elements.Moreover, the processing unit 110 may include one or more levels ofon-chip cache. Such cache elements are generally formed on the samesemiconductor die as processing unit 110 rather than being connected toprocessor subunits 120 a and 120 b via one or more buses formed in thesubstrate containing processor subunits 120 a and 120 b and the cacheelements. An arrangement directly on the same die, rather than beingconnected via buses, is common for both first-level (L1) andsecond-level (L2) caches in conventional processors. Alternatively, inolder processors, L2 caches were shared amongst processor subunits usingback-side buses between the subunits and the L2 caches. Back-side busesare generally larger than front-side buses, described below.Accordingly, because cache is to be shared with all processor subunitson the die, cache 130 may be formed on the same die as processorsubunits 120 a and 120 b or communicatively coupled to processorsubunits 120 a and 120 b via one or more back-side buses. In bothembodiments without buses (e.g., cache is formed directly on-die) aswell as embodiments using back-side buses, the caches are shared betweenprocessor subunits of the CPU.

Moreover, processing unit 110 communicates with shared memory 140 a andmemory 140 b. For example, memories 140 a and 140 b may represent memorybanks of shared dynamic random access memory (DRAM). Although depictedwith two banks, most conventional memory chips include between eight andsixteen memory banks. Accordingly, processor subunits 120 a and 120 bmay use shared memories 140 a and 140 b to store data that is thenoperated upon by processor subunits 120 a and 120 b. This arrangement,however, results in the buses between memories 140 a and 140 b andprocessing unit 110 acting as a bottleneck when the clock speeds ofprocessing unit 110 exceed data transfer speeds of the buses. This isgenerally true for conventional processors, resulting in lower effectiveprocessing speeds than the stated processing speeds based on clock rateand number of transistors.

As shown in FIG. 2, similar deficiencies persist in GPUs. A GPU 200 maycomprise a processing unit 210 that includes one or more processorsubunits (e.g., subunits 220 a, 220 b, 220 c, 220 d, 220 e, 220 f, 220g, 220 h, 220 i, 220 j, 220 k, 220 l, 220 m, 220 n, 220 o, and 220 p).Moreover, the processing unit 210 may include one or more levels ofon-chip cache and/or register files. Such cache elements are generallyformed on the same semiconductor die as processing unit 210. Indeed, inthe example of FIG. 2, cache 210 is formed on the same die as processingunit 210 and shared amongst all of the processor subunits, while caches230 a, 230 b, 230 c, and 230 d are formed on a subset of the processorsubunits, respectively, and dedicated thereto.

Moreover, processing unit 210 communicates with shared memories 250 a,250 b, 250 c, and 250 d. For example, memories 250 a, 250 b, 250 c, and250 d may represent memory banks of shared DRAM. Accordingly, theprocessor subunits of processing unit 210 may use shared memories 250 a,250 b, 250 c, and 250 d to store data that is then operated upon by theprocessor subunits. This arrangement, however, results in the busesbetween memories 250 a, 250 b, 250 c, and 250 d and processing unit 210acting as a bottleneck, similar to the bottleneck described above forCPUs.

Overview of Disclosed Hardware Chips

FIG. 3A is a diagrammatic representation of an embodiment depicting anexemplary hardware chip 300. Hardware chip 300 may comprise adistributed processor designed to mitigate the bottlenecks describedabove for CPUs, GPUs, and other conventional processors. A distributedprocessor may include a plurality of processor subunits distributedspatially on a single substrate. Moreover, as explained above, indistributed processors of the present disclosure, corresponding memorybanks are also spatially distributed on the substrate. In someembodiments, a distributed processor may be associated with a set ofinstructions, and each one of the processor subunits of the distributedprocessor may be responsible for performing one or more tasks includedin the set of instructions.

As depicted in FIG. 3A, hardware chip 300 may comprise a plurality ofprocessor subunits, e.g., logic and control subunits 320 a, 320 b, 320c, 320 d, 320 e, 320 f, 320 g, and 320 h. As further depicted in FIG.3A, each processor subunit may have a dedicated memory instance. Forexample, logic and control subunit 320 a is operably connected todedicated memory instance 330 a, logic and control subunit 320 b isoperably connected to dedicated memory instance 330 b, logic and controlsubunit 320 c is operably connected to dedicated memory instance 330 c,logic and control subunit 320 d is operably connected to dedicatedmemory instance 330 d, logic and control subunit 320 e is operablyconnected to dedicated memory instance 330 e, logic and control subunit320 f is operably connected to dedicated memory instance 330 f, logicand control subunit 320 g is operably connected to dedicated memoryinstance 330 g, and logic and control subunit 320 h is operablyconnected to dedicated memory instance 330 h.

Although FIG. 3A depicts each memory instance as a single memory bank,hardware chip 300 may include two or more memory banks as a dedicatedmemory instance for a processor subunit on hardware chip 300.Furthermore, although FIG. 3A depicts each processor subunit ascomprising both a logic component and a control for the dedicated memorybank(s), hardware chip 300 may use controls for the memory banks thatare separate, at least in part, from the logic components. Moreover, asdepicted in FIG. 3A, two or more processor subunits and theircorresponding memory banks may be grouped, e.g., into processing groups310 a, 310 b, 310 c, and 310 d. A “processing group” may represent aspatial distinction on a substrate on which hardware chip 300 is formed.Accordingly, a processing group may include further controls for thememory banks in the group, e.g., controls 340 a, 340 b, 340 c, and 340d. Additionally or alternatively, a “processing group” may represent alogical grouping for the purposes of compiling code for execution onhardware chip 300. Accordingly, a compiler for hardware chip 300(further described below) may divide an overall set of instructionsbetween the processing groups on hardware chip 300.

Furthermore, host 350 may provide instructions, data, and other input tohardware chip 300 and read output from the same. Accordingly, a set ofinstructions may be executed entirely on a single die, e.g., the diehosting hardware chip 300. Indeed, the only communications off-die mayinclude the loading of instructions to hardware chip 300, any input sentto hardware chip 300, and any output read from hardware chip 300.Accordingly, all calculations and memory operations may be performedon-die (on hardware chip 300) because the processor subunits of hardwarechip 300 communicate with dedicated memory banks of hardware chip 300.

FIG. 3B is a diagrammatic representation of an embodiment depictinganother exemplary hardware chip 300′. Although depicted as analternative to hardware chip 300, the architecture depicted in FIG. 3Bmay be combined, at least in part, with the architecture depicted inFIG. 3A.

As depicted in FIG. 3B, hardware chip 300′ may comprise a plurality ofprocessor subunits, e.g., processor subunits 350 a, 350 b, 350 c, and350 d. As further depicted in FIG. 3B, each processor subunit may have aplurality of dedicated memory instances. For example, processor subunit350 a is operably connected to dedicated memory instances 330 a and 330b, processor subunit 350 b is operably connected to dedicated memoryinstances 330 c and 330 d, processor subunit 350 c is operably connectedto dedicated memory instances 330 e and 330 f, and processor subunit 350d is operably connected to dedicated memory instances 330 g and 330 h.Moreover, as depicted in FIG. 3B, the processor subunits and theircorresponding memory banks may be grouped, e.g., into processing groups310 a, 310 b, 310 c, and 310 d. As explained above, a “processing group”may represent a spatial distinction on a substrate on which hardwarechip 300′ is formed and/or a logical grouping for the purposes ofcompiling code for execution on hardware chip 300′.

As further depicted in FIG. 3B, the processor subunits may communicatewith each other via buses. For example, as shown in FIG. 3B, processorsubunit 350 a may communicate with processor subunit 350 b via bus 360a, with processor subunit 350 c via bus 360 c, and with processorsubunit 350 d via bus 360 f. Similarly, processor subunit 350 b maycommunicate with processor subunit 350 a via bus 360 a (as describedabove), with processor subunit 350 c via bus 360 e, and with processorsubunit 350 d via bus 360 d. In addition, processor subunit 350 c maycommunicate with processor subunit 350 a via bus 360 c (as describedabove), with processor subunit 350 b via bus 360 e (as described above),and with processor subunit 350 d via bus 360 b. Accordingly, processorsubunit 350 d may communicate with processor subunit 350 a via bus 360 f(as described above), with processor subunit 350 b via bus 360 d (asdescribed above), and with processor subunit 350 c via bus 360 b (asdescribed above). One of ordinary skill will understand that fewer busesthan depicted in FIG. 3B may be used. For example, bus 360 e may beeliminated such that communications between processor subunit 350 b and350 c pass through processor subunit 350 a and/or 350 d. Similarly, bus360 f may be eliminated such that communications between processorsubunit 350 a and processor subunit 350 d pass through processor subunit350 b or 350 c.

Moreover, one of ordinary skill will understand that architectures otherthan those depicted in FIGS. 3A and 3B may be used. For example, anarray of processing groups, each with a single processor subunit andmemory instance, may be arranged on a substrate. Processor subunits mayadditionally or alternatively form part of controllers for correspondingdedicated memory banks, part of controllers for memory mats ofcorresponding dedicated memory, or the like.

In view of the architecture described above, hardware chips 300 and 300′may provide significant increases in efficiency for memory-intensivetasks as compared with traditional architectures. For example, databaseoperations and artificial intelligence algorithms (such as neuralnetworks) are examples of memory-intensive tasks for which traditionalarchitectures are less efficient than hardware chips 300 and 300′.Accordingly, hardware chips 300 and 300′ may be referred to as databaseaccelerator processors and/or artificial intelligence acceleratorprocessors.

Configuring the Disclosed Hardware Chips

The hardware chip architecture described above may be configured forexecution of code. For example, each processor subunit may individuallyexecute code (defining a set of instructions) apart from other processorsubunits in the hardware chip. Accordingly, rather than relying on anoperating system to manage multithreading or using multitasking (whichis concurrency rather than parallelism), hardware chips of the presentdisclosure may allow for processor subunits to operate fully inparallel.

In addition to a fully parallel implementation described above, at leastsome of the instructions assigned to each processor subunit may beoverlapping. For example, a plurality of processor subunits on adistributed processor may execute overlapping instructions as, forexample, an implementation of an operating system or other managementsoftware, while executing non-overlapping instructions in order toperform parallel tasks within the context of the operating system orother management software.

FIG. 4 depicts an exemplary process 400 for executing a generic commandwith processing group 410. For example, processing group 410 maycomprise a portion of a hardware chip of the present disclosure, e.g.,hardware chip 300, hardware chip 300′, or the like.

As depicted in FIG. 4, a command may be sent to processor subunit 430,which is paired with dedicated memory instance 420. An external host(e.g., host 350) may send the command to processing group 410 forexecution. Alternatively, host 350 may have sent an instruction setincluding the command for storage in memory instance 420 such thatprocessor subunit 430 may retrieve the command from memory instance 420and execute the retrieved command. Accordingly, the command may beexecuted by processing element 440, which is a generic processingelement configurable to execute the received command. Moreover,processing group 410 may include a control 460 for memory instance 420.As depicted in FIG. 4, control 460 may perform any reads and/or writesto memory instance 420 required by processing element 440 when executingthe received command After execution of the command, processing group410 may output the result of the command, e.g., to the external host orto a different processing group on the same hardware chip.

In some embodiments, as depicted in FIG. 4, processor subunit 430 mayfurther include an address generator 450. An “address generator” maycomprise a plurality of processing elements that are configured todetermine addresses in one or more memory banks for performing reads andwrites and may also perform operations on the data located at thedetermined addresses (e.g., addition, subtraction, multiplication, orthe like). For example, address generator 450 may determine addressesfor any reads or writes to memory. In one example, address generator 450may increase efficiency by overwriting a read value with a new valuedetermined based on the command when the read value is no longer needed.Additionally or alternatively, address generator 450 may selectavailable addresses for storage of results from execution of thecommand. This may allow for scheduling of result read-off for a laterclock cycle, when it is more convenient for the external host. Inanother example, address generator 450 may determine addresses to readfrom and write to during a multi-cycle calculation, such as a vector ormatrix multiply-accumulate calculation. Accordingly, address generator450 may maintain or calculate memory addresses for reading data andwriting intermediate results of the multi-cycle calculation such thatprocessor subunit 430 may continue processing without having to storethese memory addresses.

FIG. 5 depicts an exemplary process 500 for executing a specializedcommand with processing group 510. For example, processing group 510 maycomprise a portion of a hardware chip of the present disclosure, e.g.,hardware chip 300, hardware chip 300′, or the like.

As depicted in FIG. 5, a specialized command (e.g., amultiply-accumulate command) may be sent to processing element 530,which is paired with dedicated memory instance 520. An external host(e.g., host 350) may send the command to processing element 530 forexecution. Accordingly, the command may be executed at a given signalfrom the host by processing element 530, a specialized processingelement configurable to execute particular commands (including thereceived command) Alternatively, processing element 530 may retrieve thecommand from memory instance 520 for execution. Thus, in the example ofFIG. 5, processing element 530 is a multiply-accumulate (MAC) circuitconfigured to execute MAC commands received from the external host orretrieved from memory instance 520. After execution of the command,processing group 410 may output the result of the command, e.g., to theexternal host or to a different processing group on the same hardwarechip. Although depicted with a single command and a single result, aplurality of commands may be received or retrieved and executed, and aplurality of results may be combined on processing group 510 beforeoutput.

Although depicted as a MAC circuit in FIG. 5, additional or alternativespecialized circuits may be included in processing group 510. Forexample, a MAX-read command (which returns the max value of a vector) aMAX0-read command (a common function also termed a rectifier, whichreturns the entire vector but also does MAX with 0), or the like may beimplemented.

Although depicted separately, the generalized processing group 410 ofFIG. 4 and the specialized processing group 510 of FIG. 5 may becombined. For example, a generic processor subunit may be coupled to oneor more specialized processor subunits to form a processor subunit.Accordingly, the generic processor subunit may be used for allinstructions not executable by the one or more specialized processorsubunits.

One of ordinary skill will understand that neural network implementationand other memory-intensive tasks may be handled with specialized logiccircuits. For example, database queries, packet inspection, stringcomparison, and other functions may increase in efficiency if executedby the hardware chips described herein.

A Memory-Based Architecture for Distributed Processing

On hardware chips consistent with the present disclosure, dedicatedbuses may transfer data between processor subunits on the chip and/orbetween the processor subunits and their corresponding dedicated memorybanks. The use of dedicated buses may reduce arbitration costs becausecompeting requests are either not possible or easily avoided usingsoftware rather than hardware.

FIG. 6 schematically depicts a diagrammatic representation of aprocessing group 600. Processing group 600 may be for use in a hardwarechip, e.g., hardware chip 300, hardware chip 300′, or the like.Processor subunit 610 may be connected via buses 630 to memory 620.Memory 620 may comprise a Randomly Accessible Memory (RAM) element thatstores data and code for execution by processor subunit 610. In someembodiments, memory 620 may be an N-Way memory (wherein N is a numberequal to or larger than 1 that implies the number of segments in aninterleaved memory 620). Because processor subunit 610 is coupled tomemory 620 dedicated to processor subunit 610 via bus 630, N may be keptrelatively small without compromising the execution performance. Thisrepresents an improvement over conventional multiway register files orcaches where a lower N generally results in lower execution performance,and a higher N generally results in large area and power loss.

The size of memory 620, the number of ways, and the width of bus 630 maybe adjusted to meet the requirements of tasks and applicationimplementations of a system using processing group 600 according to, forinstance, the size of data involved in the task or tasks. Memory element620 may comprise one or more types of memory known in the art, e.g.,volatile memory (such as RAM, DRAM, SRAM, phase-change RAM (PRAM),magnetoresistive RAM (MRAM), resistive RAM (ReRAM), or the like) ornon-volatile memory (such as flash or ROM). According to someembodiments, a portion of memory element 620 may comprise a first memorytype, while another portion may comprise another memory type. Forinstance, the code region of a memory element 620 may comprise a ROMelement, while a data region of the memory element 620 may comprise aDRAM element. Another example for such partitioning is storing theweights of a neural network in flash while storing the data forcalculation in DRAM.

Processor subunit 610 comprises a processing element 640 that maycomprise a processor. The processor can be pipelined or not pipelined, acustomized Reduced Instruction Set Computing (RISC) element or otherprocessing scheme, implemented on any commercial Integrated Circuit (IC)known in the art (such as ARM, ARC, RISC-V, etc.), as appreciated by oneof ordinary skill. Processing element 640 may comprise a controllerthat, in some embodiments, includes an Arithmetic Logic Unit (ALU) orother controller.

According to some embodiments, processing element 640, which executesreceived or stored code, may comprise a generic processing element and,therefore, be flexible and capable of performing a wide variety ofprocessing operations. Non-dedicated circuitry typically consumes morepower than specific-operation-dedicated circuitry when comparing thepower consumed during performance for a specific operation. Therefore,when performing specific complex arithmetic calculations, processingelement 640 may consume more power and perform less efficiently thandedicated hardware. Therefore, according to some embodiments, acontroller of processing element 640 may be designed to perform specificoperations (e.g., addition or “move” operations).

In one example, the specific operations may be performed by one or moreaccelerators 650. Each accelerator may be dedicated and programmed toperform a specific calculation (such as multiplication, floating pointvector operations, or the like). By using accelerator(s), the averagepower consumed per calculation per processor subunit may be lowered, andthe calculation throughput henceforth increases. Accelerator(s) 650 maybe chosen according to an application that the system is designed toimplement (e.g., execution of neural networks, execution of databasequeries, or the like). Accelerator(s) 650 may be configured byprocessing element 640 and may operate in tandem therewith for loweringpower consumption and accelerating calculations and computations. Theaccelerators may additionally or alternatively be used to transfer databetween memory and MUXs/DEMUXs/input/output ports (e.g., MUX 650 andDEMUX 660) of processing group 600, such as a smart DMA (direct memoryaccess) peripheral.

Accelerator(s) 650 may be configured to perform a variety of functions.For instance, one accelerator may be configured to perform 16-bitfloating point calculation or 8-bit integer calculations, which areoften used in neural networks. Another example of an acceleratorfunction is a 32-bit floating point calculation, which is often usedduring a training stage of a neural network. Yet another example of anaccelerator function is query processing, such as that used indatabases. In some embodiments, accelerator(s) 650 may comprisespecialized processing elements to perform these functions and/or may beconfigured according to configuration data, stored on the memory element620, such that it may be modified.

Accelerator(s) 650 may additionally or alternatively implement aconfigurable scripted list of memory movements to time movements of datato/from memory 620 or to/from other accelerators and/or inputs/outputs.Accordingly, as explained further below, all the data movement insidethe hardware chip using processing group 600 may use softwaresynchronization rather than hardware synchronization. For example, anaccelerator in one processing group (e.g., group 600) may transfer datafrom its input to its accelerator every tenth cycle and then output dataat the next cycle, thereby letting the information flow from the memoryof the processing group to another one.

As further depicted in FIG. 6, in some embodiments, processing group 600may further comprise at least one input multiplexer (MUX) 660 connectedto its input port and at least one output DEMUX 670 connected to itsoutput port. These MUXs/DEMUXs may be controlled by control signals (notshown) from processing element 640 and/or from one of accelerator(s)650, determined according to a current instruction being carried out byprocessing element 640 and/or the operation executed by an acceleratorof accelerator(s) 650. In some scenarios, processing group 600 may berequired (according to a predefined instruction from its code memory) totransfer data from its input port to its output port. Accordingly, oneor more of the input MUXs (e.g., MUX 660) may be directly connected viaone or more buses to an output DEMUX (e.g., DEMUX 670), in addition toeach of the DEMUXs/MUXs being connected to processing element 640 andaccelerator(s) 650.

The processing group 600 of FIG. 6 may be arrayed to form a distributedprocessor, for example, as depicted in FIG. 7A. The processing groupsmay be disposed on substrate 710 to form an array. In some embodiments,substrate 710 may comprise a semiconductor substrate, such as silicon.Additionally or alternatively, substrate 710 may comprise a circuitboard, such as a flexible circuit board.

As depicted in FIG. 7A, substrate 710 may include, disposed thereon, aplurality of processing groups, such as processing group 600.Accordingly, substrate 710 includes a memory array that includes aplurality of banks, such as banks 720 a, 720 b, 720 c, 720 d, 720 e, 720f, 720 g, and 720 h. Furthermore, substrate 710 includes a processingarray that may include a plurality of processor subunits, such assubunits 730 a, 730 b, 730 c, 730 d, 730 e, 730 f, 730 g, and 730 h.

Furthermore, as explained above, each processing group may include aprocessor subunit and one or more corresponding memory banks dedicatedto the processor subunit. Accordingly, as depicted in FIG. 7A, eachsubunit is associated with a corresponding, dedicated memory bank, e.g.:Processor subunit 730 a is associated with memory bank 720 a, processorsubunit 730 b is associated with memory bank 720 b, processor subunit730 c is associated with memory bank 720 c, processor subunit 730 d isassociated with memory bank 720 d, processor subunit 730 e is associatedwith memory bank 720 e, processor subunit 730 f is associated withmemory bank 720 f, processor subunit 730 g is associated with memorybank 720 g, processor subunit 730 h is associated with memory bank 720h.

To allow each processor subunit to communicate with its corresponding,dedicated memory bank(s), substrate 710 may include a first plurality ofbuses connecting one of the processor subunits to its corresponding,dedicated memory bank(s). Accordingly, bus 740 a connects processorsubunit 730 a to memory bank 720 a, bus 740 b connects processor subunit730 b to memory bank 720 b, bus 740 c connects processor subunit 730 cto memory bank 720 c, bus 740 d connects processor subunit 730 d tomemory bank 720 d, bus 740 e connects processor subunit 730 e to memorybank 720 e, bus 740 f connects processor subunit 730 f to memory bank720 f, bus 740 g connects processor subunit 730 g to memory bank 720 g,and bus 740 h connects processor subunit 730 h to memory bank 720 h.Moreover, to allow each processor subunit to communicate with otherprocessor subunits, substrate 710 may include a second plurality ofbuses connecting one of the processor subunits to another of theprocessor subunits. In the example of FIG. 7A, bus 750 a connectsprocessor subunit 730 a to processor subunit 750 e, bus 750 b connectsprocessor subunit 730 a to processor subunit 750 b, bus 750 c connectsprocessor subunit 730 b to processor subunit 750 f, bus 750 d connectsprocessor subunit 730 b to processor subunit 750 c, bus 750 e connectsprocessor subunit 730 c to processor subunit 750 g, bus 750 f connectsprocessor subunit 730 c to processor subunit 750 d, bus 750 g connectsprocessor subunit 730 d to processor subunit 750 h, bus 750 h connectsprocessor subunit 730 h to processor subunit 750 g, bus 750 i connectsprocessor subunit 730 g to processor subunit 750 g, and bus 750 jconnects processor subunit 730 f to processor subunit 750 e.

Accordingly, in the example arrangement shown in FIG. 7A, the pluralityof logic processor subunits is arranged in at least one row and at leastone column. The second plurality of buses connect each processor subunitto at least one adjacent processor subunit in the same row and to atleast one adjacent processor subunit in the same column. FIG. 7A may bereferred to as a “partial tile connection.”

The arrangement shown in FIG. 7A may be modified to form a “full tileconnection.” A full tile connection includes additional buses connectingdiagonal processor subunits. For example, the second plurality of busesmay include additional buses between processor subunit 730 a andprocessor subunit 730 f, between processor subunit 730 b and processorsubunit 730 e, between processor subunit 730 b and processor subunit 730g, between processor subunit 730 c and processor subunit 730 f, betweenprocessor subunit 730 c and processor subunit 730 h, and betweenprocessor subunit 730 d and processor subunit 730 g.

A full tile connection may be used for convolution calculations, inwhich data and results stored in a near processor subunit are used. Forexample, during convolutional image processing, each processor subunitmay receive a tile of the image (such as a pixel or a group of pixels).In order to calculate the convolution results, each processor subunitmay acquire data from all eight adjacent processor subunits, each ofwhich have received a corresponding tile. In a partial tile connection,the data from the diagonal adjacents may be passed through otheradjacent processor subunits connected to the processor subunit.Accordingly, the distributed processor on a chip may be an artificialintelligence accelerator processor.

In a specific example of a convolutional calculation, an N×M image maybe divided across a plurality of processor subunits. Each processorsubunit may perform a convolution with an A×B filter on itscorresponding tile. To perform the filtering on one or more pixels on aboundary between tiles, each processor subunit may require data fromneighboring processor subunits having tiles including pixels on the sameboundary. Accordingly, the code generated for each processor subunitconfigures the subunit to calculate the convolutions and pull from oneof the second plurality of buses whenever data is needed from anadjacent subunit. Corresponding commands to output data to the secondplurality of buses are provided to the subunits to ensure proper timingof needed data transfers.

The partial tile connection of FIG. 7A may be modified to be anN-partial tile connection. In this modification, the second plurality ofbuses may further connect each processor subunit to processor subunitswithin a threshold distance of the processor subunit (e.g., within nprocessor subunits) in the four directions along which the buses of FIG.7A run (i.e., up, down, left, and right). A similar modification may bemade to the full-tile connection (to result in an N-full tileconnection) such that the second plurality of buses further connectseach processor subunit to processor subunits within a threshold distanceof the processor subunit (e.g., within n processor subunits) in the fourdirections along which the buses of FIG. 7A run in additional to the twodiagonal directions.

Other arrangements are possible. For example, in the arrangement shownin FIG. 7B, bus 750 a connects processor subunit 730 a to processorsubunit 730 d, bus 750 b connects processor subunit 730 a to processorsubunit 730 b, bus 750 c connects processor subunit 730 b to processorsubunit 730 c, and bus 750 d connects processor subunit 730 c toprocessor subunit 730 d. Accordingly, in the example arrangement shownin FIG. 7B, the plurality of processor subunits is arranged in a starpattern. The second plurality of buses connect each processor subunit toat least one adjacent processor subunit within the star pattern.

Further arrangements (not shown) are possible. For example, a neighborconnection arrangement may be used such that the plurality of processorsubunits is arranged in one or more lines (e.g., similar to thatdepicted in FIG. 7A). In a neighbor connection arrangement, the secondplurality of buses connect each processor subunit to a processor subunitto the left in the same line, to a processor subunit to the right in thesame line, to the processor subunits both to the left and to the rightin the same line, etc.

In another example, an N-linear connection arrangement may be used. Inan N-linear connection arrangement, the second plurality of busesconnect each processor subunit to processor subunits within a thresholddistance of the processor subunit (e.g., within n processor subunits).The N-linear connection arrangement may be used with the line array(described above), the rectangular array (depicted in FIG. 7A), theelliptical array (depicted in FIG. 7B), or any other geometrical array.

In yet another example, an N-log connection arrangement may be used. Inan N-log connection arrangement, the second plurality of buses connecteach processor subunit to processor subunits within a threshold power oftwo distance of the processor subunit (e.g., within T processorsubunits). The N-log connection arrangement may be used with the linearray (described above), the rectangular array (depicted in FIG. 7A),the elliptical array (depicted in FIG. 7B), or any other geometricalarray.

Any of the connection schemes described above may be combined for use inthe same hardware chip. For example, a full tile connection may be usedin one region while a partial tile connection is used in another region.In another example, an N-linear connection arrangement may be used inone region while an N-full tile connection is used in another region.

Alternatively to or in addition with dedicated buses between processorsubunits of the memory chip, one or more shared buses may be used tointerconnect all (or a subset of) the processor subunits of adistributed processor. Collisions on the shared buses may still beavoided by timing data transfers on the shared buses using code executedby the processor subunits, as explained further below. Additionally withor alternatively to shared buses, configurable buses may be used todynamically connect processor subunits to form groups of processorsunits connected to separated buses. For example, the configurable busesmay include transistors or other mechanisms that may be controlled byprocessor subunit to direct data transfers to a selected processorsubunit.

In both FIGS. 7A and 7B, the plurality of processor subunits of theprocessing array is spatially distributed among the plurality ofdiscrete memory banks of the memory array. In other alternativeembodiments (not shown), the plurality of processor subunits may beclustered in one or more regions of the substrate, and the plurality ofmemory banks may be clustered in one or more other regions of thesubstrate. In some embodiments, a combination of spatial distributionand clustering may be used (not shown). For example, one region of thesubstrate may include a cluster of processor subunits, another region ofthe substrate may include a cluster of memory banks, and yet anotherregion of the substrate may include processing arrays distributedamongst memory banks.

One of ordinary skill will recognize that arraying processor groups 600on a substrate is not an exclusive embodiment. For example, eachprocessor subunit may be associated with at least two dedicated memorybanks. Accordingly, processing groups 310 a, 310 b, 310 c, and 310 d ofFIG. 3B may be used in lieu of or in combination with processing group600 to form the processing array and the memory array. Other processinggroups including, for example, three, four, or more dedicated memorybanks (not shown) may be used.

Each of the plurality of processor subunits may be configured to executesoftware code associated with a particular application independently,relative to other processor subunits included in the plurality ofprocessor subunits. For example, as explained below, a plurality ofsub-series of instructions may be grouped as machine code and providedto each processor subunit for execution.

In some embodiments, each dedicated memory bank comprises at least onedynamic random access memory (DRAM). Alternatively, the memory banks maycomprise a mix of memory types, such as static random access memory(SRAM), DRAM, Flash or the like.

In conventional processors, data sharing between processor subunits isusually performed with shared memory. Shared memory typically requires alarge portion of chip area and/or performed a bus that is managed byadditional hardware (such as arbiters). The bus results in bottlenecks,as described above. In addition, the shared memory, which may beexternal to the chip, typically includes cache coherency mechanisms andmore complex caches (e.g., L1 cache, L2 cache, and shared DRAM) in orderto provide accurate and up-to-date data to the processor subunits. Asexplained further below, the dedicated buses depicted in FIGS. 7A and 7Ballow for hardware chips that are free of hardware management (such asarbiters). Moreover, the use of dedicated memories as depicted in FIGS.7A and 7B allow for the elimination of complex caching layers andcoherency mechanism.

Instead, in order to allow each processor subunit to access datacalculated by other processor subunits and/or stored in memory banksdedicated to the other processor subunits, buses are provided whosetiming is performed dynamically using code individually executed by eachprocessor subunit. This allows for elimination of most, if not all, busmanagement hardware as conventionally used. Moreover, complex cachingmechanisms are replaced with direct transfers over these buses,resulting in lower latency times during memory reads and writes.

Memory-Based Processing Arrays

As depicted in FIGS. 7A and 7B, a memory chip of the present disclosuremay operate independently. Alternatively, memory chips of the presentdisclosure may be operably connected with one or more additionalintegrated circuits, such as a memory device (e.g., one or more DRAMbanks), a system-on-a-chip, a field-programmable gate array (FPGA), orother processing and/or memory chip. In such embodiments, tasks in aseries of instructions executed by the architecture may be divided(e.g., by a compiler, as described below) between processor subunits ofthe memory chip and any processor subunits of the additional integratedcircuit(s). For example, the other integrated circuits may comprise ahost (e.g., host 350 of FIG. 3A) that inputs instructions and/or data tothe memory chip and receives output therefrom.

In order to interconnect memory chips of the present disclosure with oneor more additional integrated circuits, the memory chip may include amemory interface, such as a memory interface complying with a JointElectron Device Engineering Council (JEDEC) standard or any of itsvariants. The one or more additional integrated circuits may thenconnect to the memory interface. Accordingly, if the one or moreadditional integrated circuits are connected to a plurality of memorychips of the present disclosure, data may be shared between the memorychips through the one or more additional integrated circuits.Additionally or alternatively, the one or more additional integratedcircuits may include buses to connect to buses on the memory chips ofthe present disclosure such that the one or more additional integratedcircuits may execute code in tandem with the memory chips of the presentdisclosure. In such embodiments, the one or more additional integratedcircuits further assist with distributed processing even though they maybe on different substrates than the memory chips of the presentdisclosure.

Furthermore, memory chips of the present disclosure may be arrayed inorder to form an array of distributed processors. For example, one ormore buses may connect a memory chip 770 a to an additional memory chip770 b, as depicted in FIG. 7C. In the example of FIG. 7C, memory chip770 a includes processor subunits with one or more corresponding memorybanks dedicated to each processor subunit, e.g.: Processor subunit 730 ais associated with memory bank 720 a, processor subunit 730 b isassociated with memory bank 720 b, processor subunit 730 e is associatedwith memory bank 720 c, and processor subunit 730 f is associated withmemory bank 720 d. Buses connect each processor subunit to itscorresponding memory bank. Accordingly, bus 740 a connects processorsubunit 730 a to memory bank 720 a, bus 740 b connects processor subunit730 b to memory bank 720 b, bus 740 c connects processor subunit 730 eto memory bank 720 c, and bus 740 d connects processor subunit 730 f tomemory bank 720 d. Moreover, bus 750 a connects processor subunit 730 ato processor subunit 750 e, bus 750 b connects processor subunit 730 ato processor subunit 750 b, bus 750 c connects processor subunit 730 bto processor subunit 750 f, and bus 750 d connects processor subunit 730e to processor subunit 750 f. Other arrangements of memory chip 770 amay be used, for example, as described above.

Similarly, memory chip 770 b includes processor subunits with one ormore corresponding memory banks dedicated to each processor subunit,e.g.: Processor subunit 730 c is associated with memory bank 720 e,processor subunit 730 d is associated with memory bank 720 f, processorsubunit 730 g is associated with memory bank 720 g, and processorsubunit 730 h is associated with memory bank 720 h. Buses connect eachprocessor subunit to its corresponding memory bank. Accordingly, bus 740e connects processor subunit 730 c to memory bank 720 e, bus 740 fconnects processor subunit 730 d to memory bank 720 f, bus 740 gconnects processor subunit 730 g to memory bank 720 g, and bus 740 hconnects processor subunit 730 h to memory bank 720 h. Moreover, bus 750g connects processor subunit 730 c to processor subunit 750 g, bus 750 hconnects processor subunit 730 d to processor subunit 750 h, bus 750 iconnects processor subunit 730 c to processor subunit 750 d, and bus 750j connects processor subunit 730 g to processor subunit 750 h. Otherarrangements of memory chip 770 b may be used, for example, as describedabove.

The processor subunits of memory chip 770 a and 770 b may be connectedusing one or more buses. Accordingly, in the example of FIG. 7C, bus 750e may connect processor subunit 730 b of memory chip 770 a and processorsubunit 730 c of memory chip 770 b, and bus 750 f may connect processorsubunit 730 f of memory chip 770 a and processor subunit 730 c of memory770 b. For example, bus 750 e may serve as an input bus to memory chip770 b (and thus an output bus for memory chip 770 a) while bus 750 f mayserve as an input bus to memory chip 770 a (and thus an output bus formemory chip 770 b) or vice versa. Alternatively, buses 750 e and 750 fmay both server as two-way buses between memory chips 770 a and 770 b.

Buses 750 e and 750 f may include direct wires or may be interleaved ona high-speed connection in order to reduce the pins used for theinter-chip interface between memory chip 770 a and integrated circuit770 b. Moreover, any of the connection arrangements described above usedin the memory chip itself may be used to connect the memory chip to oneor more additional integrated circuits. For example, memory chip 770 aand 770 b may be connected using a full-tile or partial-tile connectionrather than only two buses as shown in FIG. 7C.

Accordingly, although depicted using buses 750 e and 750 f, architecture760 may include fewer buses or additional buses. For example, a singlebus between processor subunits 730 b and 730 c or between processorsubunits 730 f and 730 c may be used. Alternatively, additional buses,e.g., between processor subunits 730 b and 730 d, between processorsubunits 730 f and 730 d, or the like, may be used.

Furthermore, although depicted as using a single memory chip and anadditional integrated circuit, a plurality of memory chips may beconnected using buses as explained above. For example, as depicted inthe example of FIG. 7C, memory chips 770 a, 770 b, 770 c, and 770 d areconnected in an array. Each memory chip includes processor subunits anddedicated memory banks similar to the memory chips described above.Accordingly, a description of these components is not repeated here.

In the example of FIG. 7C, memory chips 770 a, 770 b, 770 c, and 770 dare connected in a loop. Accordingly, bus 750 a connects memory chips770 a and 770 d, bus 750 c connects memory chips 770 a and 770 b, bus750 e connects memory chips 770 b and 770 c, and bus 750 g connectsmemory chips 770 c and 770 d. Although memory chips 770 a, 770 b, 770 c,and 770 d may be connected with full-tile connections, partial-tileconnections, or other connection arrangements, the example of FIG. 7Callows for fewer pin connections between memory chips 770 a, 770 b, 770c, and 770 d.

Relatively Large Memories

Embodiments of the present disclosure may use dedicated memories ofrelatively large size as compared with shared memories of conventionalprocessors. The use of dedicated memories rather than shared memoriesallows for gains in efficiency to continue without tapering off withmemory increases. This allows for memory-intensive tasks such as neuralnetwork processing and database queries to be performed more efficientlythan in conventional processors, where the efficiency gains ofincreasing shared memory taper off due to the von Neumann bottleneck.

For example, in distributed processors of the present disclosure, amemory array disposed on the substrate of the distributed processor mayinclude a plurality of discrete memory banks Each of the discrete memorybanks may have a capacity greater than one megabyte, as well as aprocessing array disposed on the substrate, including a plurality ofprocessor subunits. As explained above, each one of the processorsubunits may be associated with a corresponding, dedicated one of theplurality of discrete memory banks In some embodiments, the plurality ofprocessor subunits may be spatially distributed among the plurality ofdiscrete memory banks within the memory array. By using dedicatedmemories of at least one megabyte, rather than shared caches of a fewmegabytes for a large CPU or GPU, the distributed processors of thepresent disclosure gain efficiencies that are not possible inconventional systems due to the von Neumann bottleneck in CPUs and GPUs.

Different memories may be used as the dedicated memories. For example,each dedicated memory bank may comprise at least one DRAM bank.Alternatively, each dedicated memory bank may comprise at least onestatic random access memory bank. In other embodiments, different typesof memories may be combined on a single hardware chip.

As explained above, each dedicated memory may be at least one megabyte.Accordingly, each dedicated memory bank may be the same size or at leasttwo of the plurality of memory banks may have different sizes.

Moreover, as described above, the distributed processor may include afirst plurality of buses, each connecting one of the plurality ofprocessor subunits to a corresponding, dedicated memory bank and asecond plurality of buses, each connecting one of the plurality ofprocessor subunits to another one of the plurality of processorsubunits.

Synchronization Using Software

As explained above, hardware chips of the present disclosure may managedata transfers using software rather than hardware. In particular,because the timings of transfers on the buses, reads and writes to thememories, and calculations of the processor subunits are set by thesub-series of instructions executed by the processor subunits, hardwarechips of the present disclosure may execute code to prevent collisionson the buses. Accordingly, hardware chips of the present disclosure mayavoid hardware mechanisms conventionally used to manage data transfers(such as network controllers within in a chip, packet parsers andpackets transferors between processor subunits, bus arbitrators, aplurality of buses to avoid arbitration, or the like).

If hardware chips of the present disclosure transferred dataconventionally, connecting N processor subunits with buses would requirebus arbitration or wide MUXs controlled by an arbiter. Instead, asdescribed above, embodiments of the present disclosure may use a busthat is only a wire, an optical cable, or the like between processorsubunits, where the processor subunits individually execute code toavoid collision on the buses. Accordingly, embodiments of the presentdisclosure may preserve space on the substrate as well as materials costand efficiency losses (e.g., due to power and time consumption byarbitration). The efficiency and space gains are even greater whencompared to other architectures using first-in-first-out (FIFO)controllers and/or mailboxes.

Furthermore, as explained above, each processor subunit may include oneor more accelerators in addition to one or more processing elements. Insome embodiments, the accelerator(s) may read and write from the busesrather than the processing element(s). In such embodiments, additionalefficiency may be obtained by allowing the accelerator(s) to transmitdata during the same cycle in which the processing element(s) performone or more calculations. Such embodiments, however, require additionalmaterials for the accelerator(s). For example, additional transistorsmay be required for fabrication of the accelerator(s).

The code also may account for the internal behavior, including timingand latencies, of the processor subunits (e.g., including the processingelements and/or accelerators forming part of the processor subunit). Forexample, a compiler (as described below) may perform pre-processing thataccounts for the timing and latencies when generating the sub-series ofinstructions that control the data transfers.

In one example, a plurality of processor subunits may be assigned a taskof calculating a neural network layer containing a plurality of neuronsfully-connected to a previous layer of a larger plurality of neurons.Assuming data of the previous layer is evenly spread between theplurality of processor subunits, one way to perform the calculation maybe to configure each processor subunit to transmit the data of theprevious layer to the main bus in turn and then each processor subunitwill multiply this data by the weight of the corresponding neuron thatthe subunit implements. Because each processor subunit calculates morethan one neuron, each processor subunit will transmit the data of theprevious layer a number of times equal to the number of neurons. Thus,the code of each processor subunit is not the same as the code for otherprocessor subunits because the subunits will transmit at differenttimes.

In some embodiments, a distributed processor may comprise a substrate(e.g., a semiconductor substrate, such as silicon and/or a circuitboard, such as a flexible circuit board) with a memory array disposed onthe substrate, the memory array including a plurality of discrete memorybanks, and a processing array disposed on the substrate, the processingarray including a plurality of processor subunits, as depicted, e.g., inFIGS. 7A and 7B. As explained above, each one of the processor subunitsmay be associated with a corresponding, dedicated one of the pluralityof discrete memory banks. Moreover, as depicted, e.g., in FIGS. 7A and7B, the distributed processor may further comprise a plurality of buses,each one of the plurality of buses connecting one of the plurality ofprocessor subunits to at least another one of the plurality of processorsubunits.

As explained above, the plurality of buses may be controlled insoftware. Accordingly, the plurality of buses may be free of timinghardware logic components such that data transfers between processorsubunits and across corresponding ones of the plurality of buses areuncontrolled by timing hardware logic components. In one example, theplurality of buses may be free of bus arbiters such that data transfersbetween processor subunits and across corresponding ones of theplurality of buses are uncontrolled by bus arbiters.

In some embodiments, as depicted, e.g., in FIGS. 7A and 7B, thedistributed processor may further comprise a second plurality of busesconnecting one of the plurality of processor subunits to acorresponding, dedicated memory bank. Similar to the plurality of busesdescribed above, the second plurality of buses may be free of timinghardware logic components such that data transfers between processorsubunits and corresponding, dedicated memory banks are uncontrolled bytiming hardware logic components. In one example, the second pluralityof buses may be free of bus arbiters such that data transfers betweenprocessor subunits and corresponding, dedicated memory banks areuncontrolled by bus arbiters.

As used herein, the phrase “free of” does not necessarily imply theabsolute absence of components, such as timing hardware logic components(e.g., bus arbiters, arbitration trees, FIFO controllers, mailboxes, orthe like). Such components may still be included in a hardware chipdescribed as “free of” those components. Instead, the phrase “free of”refers to the function of the hardware chip; that is, a hardware chip“free of” timing hardware logic components controls the timing of itsdata transfers without use of the timing hardware logic components, ifany, included therein. For example, a hardware chip that executes codeincluding sub-series of instructions that control data transfers betweenprocessor subunits of the hardware chip, even if the hardware chipincludes timing hardware logic components as a secondary precaution toprotect against collisions due to errors in the executed code.

As explained above, the plurality of buses may comprise at least one ofwires or optical fibers between corresponding ones of the plurality ofprocessor subunits. Accordingly, in one example, a distributed processorfree of timing hardware logic components may include only wires oroptical fibers without bus arbiters, arbitration trees, FIFOcontrollers, mailboxes, or the like.

In some embodiments, the plurality of processor subunits is configuredto transfer data across at least one of the plurality of buses inaccordance with code executed by the plurality of processor subunits.Accordingly, as explained below, a compiler may organize sub-series ofinstructions, each sub-series comprising code executed by a singleprocessor subunit. The sub-series instructions may instruct theprocessor subunit when to transfer data onto one of the buses and whento retrieve data from the buses. When the sub-series are executed intandem across the distributed processor, the timing of transfers betweenthe processor subunits may be governed by the instructions to transferand retrieve included in the sub-series. Thus, the code dictates timingof data transfers across at least one of the plurality of buses. Thecompiler may generate code to be executed by a single processor subunit.Additionally, the compiler may generate code to be executed by groups ofprocessor subunits. In some cases, the compiler may treat all theprocessor subunits together as if they were one super-processor (e.g., adistributed processor), and the compiler may generate code for executionby that defined super-processor/distributed processor.

As explained above and depicted in FIGS. 7A and 7B, the plurality ofprocessor subunits may be spatially distributed among the plurality ofdiscrete memory banks within the memory array. Alternatively, theplurality of processor subunits may be clustered in one or more regionsof the substrate, and the plurality of memory banks may be clustered inone or more other regions of the substrate. In some embodiments, acombination of spatial distribution and clustering may be used, asexplained above.

In some embodiments, a distributed processor may comprise a substrate(e.g., a semiconductor substrate, including silicon and/or a circuitboard, such as a flexible circuit board) with a memory array disposed onthe substrate, the memory array including a plurality of discrete memorybanks A processing array may also be disposed on the substrate, theprocessing array including a plurality of processor subunits, asdepicted, e.g., in FIGS. 7A and 7B. As explained above, each one of theprocessor subunits may be associated with a corresponding, dedicated oneof the plurality of discrete memory banks. Moreover, as depicted, e.g.,in FIGS. 7A and 7B, the distributed processor may further comprise aplurality of buses, each one of the plurality of buses connecting one ofthe plurality of processor subunits to a corresponding, dedicated one ofthe plurality of discrete memory banks.

As explained above, the plurality of buses may be controlled insoftware. Accordingly, the plurality of buses may be free of timinghardware logic components such that data transfers between a processorsubunit and a corresponding, dedicated one of the plurality of discretememory banks and across a corresponding one of the plurality of busesare not controlled by timing hardware logic components. In one example,the plurality of buses may be free of bus arbiters such that datatransfers between processor subunits and across corresponding ones ofthe plurality of buses are uncontrolled by bus arbiters.

In some embodiments, as depicted, e.g., in FIGS. 7A and 7B, thedistributed processor may further comprise a second plurality of busesconnecting one of the plurality of processor subunits to at leastanother one of the plurality of processor subunits Similar to theplurality of buses described above, the second plurality of buses may befree of timing hardware logic components such that data transfersbetween processor subunits and corresponding, dedicated memory banks areuncontrolled by timing hardware logic components. In one example, thesecond plurality of buses may be free of bus arbiters such that datatransfers between processor subunits and corresponding, dedicated memorybanks are uncontrolled by bus arbiters.

In some embodiments, the distributed processor may use a combination ofsoftware timing with hardware timing components. For example, adistributed processor may comprise a substrate (e.g., a semiconductorsubstrate, including silicon and/or a circuit board, such as a flexiblecircuit board) with a memory array disposed on the substrate, the memoryarray including a plurality of discrete memory banks A processing arraymay also be disposed on the substrate, the processing array including aplurality of processor subunits, as depicted, e.g., in FIGS. 7A and 7B.As explained above, each one of the processor subunits may be associatedwith a corresponding, dedicated one of the plurality of discrete memorybanks. Moreover, as depicted, e.g., in FIGS. 7A and 7B, the distributedprocessor may further comprise a plurality of buses, each one of theplurality of buses connecting one of the plurality of processor subunitsto at least another one of the plurality of processor subunits.Moreover, as explained above, the plurality of processor subunits may beconfigured to execute software that controls timing of data transfersacross the plurality of buses to avoid colliding data transfers on atleast one of the plurality of buses. In such an example, the softwaremay control the timing of the data transfers, but the transfersthemselves may be controlled, at least in part, by one or more hardwarecomponents.

In such embodiments, the distributed processor may further comprise asecond plurality of buses connecting one of the plurality of processorsubunits to a corresponding, dedicated memory bank Similar to theplurality of buses described above, the plurality of processor subunitsmay be configured to execute software that controls timing of datatransfers across the second plurality of buses to avoid colliding datatransfers on at least one of the second plurality of buses. In such anexample, as explained above, the software may control the timing of thedata transfers, but the transfers themselves may be controlled, at leastin part, by one or more hardware components.

Division of Code

As explained above, hardware chips of the present disclosure may executecode in parallel across processor subunits included on a substrateforming the hardware chip. Additionally, hardware chips of the presentdisclosure may perform multitasking. For example, hardware chips of thepresent disclosure may perform area multitasking, in which one group ofprocessor subunits of the hardware chip execute one task (e.g., audioprocessing) while another group of processor subunits of the hardwarechip execute another task (e g, image processing). In another example,hardware chips of the present disclosure may perform timingmultitasking, in which one or more processor subunits of the hardwarechip execute one task during a first period of time and another taskduring a second period of time. A combination of area and timingmultitasking may also be used such that one task may be assigned to afirst group of processor subunits during a first period of time whileanother task may be assigned to a second group of processor subunitsduring the first period of time, after which a third task may beassigned to processor subunits included in the first group and thesecond group during a second period of time.

In order to organize machine code for execution on memory chips of thepresent disclosure, machine code may be divided between processorsubunits of the memory chip. For example, a processor on a memory chipmay comprise a substrate and a plurality of processor subunits disposedon the substrate. The memory chip may further comprise a correspondingplurality of memory banks disposed on the substrate, each one of theplurality processor subunits being connected to at least one dedicatedmemory bank not shared by any other processor subunit of the pluralityof processor subunits. Each processor subunit on the memory chip may beconfigured to execute a series of instructions independent from otherprocessor subunits. Each series of instructions may be executed byconfiguring one or more general processing elements of the processorsubunit in accordance with code defining the series of instructionsand/or by activating one or more special processing elements (e.g., oneor more accelerators) of the processor subunit in accordance with asequence provided in the code defining the series of instructions.

Accordingly, each series of instructions may define a series of tasks tobe performed by a single processor subunit. A single task may comprisean instruction within an instruction set defined by the architecture ofone or more processing elements in the processor subunit. For example,the processor subunit may include particular registers, and a singletask may push data onto a register, pull data from a register, performan arithmetic function on data within a register, perform a logicoperation on data within a register, or the like. Moreover, theprocessor subunit may be configured for any number of operands, such asa 0-operand processor subunit (also called a “stack machine”), a1-operand processor subunit (also called an accumulator machine), a2-operand processor subunit (such as a RISC), a 3-operand processorsubunit (such as a complex instruction set computer (CISC)), or thelike. In another example, the processor subunit may include one or moreaccelerators, and a single task may activate an accelerator to perform aspecific function, such as a MAC function, a MAX function, a MAX-0function, or the like.

The series of instructions may further include tasks for reading andwriting from the dedicated memory banks of the memory chip. For example,a task may include writing a piece of data to a memory bank dedicated tothe processor subunit executing the task, reading a piece of data from amemory bank dedicated to the processor subunit executing the task, orthe like. In some embodiments, the reading and writing may be performedby the processor subunit in tandem with a controller of the memory bank.For example, the processor subunit may execute a read or write task bysending a control signal to the controller to perform the read or write.In some embodiments, the control signal may include a particular addressto use for reads and writes. Alternatively, the processor subunit maydefer to the memory controller to select an available address for thereads and writes.

Additionally or alternatively, the reading and writing may be performedby one or more accelerators in tandem with a controller of the memorybank. For example, the accelerators may generate the control signals forthe memory controller, similar to how the processor subunit generatescontrol signals, as described above.

In any of the embodiments described above, an address generator may alsobe used to direct the reads and writes to specific addresses of a memorybank. For example, the address generator may comprise a processingelement configured to generate memory addresses for reads and writes.The address generator may be configured to generate addresses in orderto increase efficiency, e.g., by writing results of a later calculationto the same address as the results of a former calculation that are nolonger needed. Accordingly, the address generator may generate thecontrols signals for the memory controller, either in response to acommand from the processor subunit (e.g., from a processing elementincluded therein or from one or more accelerator(s) therein) or intandem with the processor subunit. Additionally or alternatively, theaddress generator may generate the addresses based on some configurationor registers for example generating a nested loop structure to iterateon certain addresses in the memory at a certain pattern.

In some embodiments, each series of instructions may comprise a set ofmachine code defining a corresponding series of tasks. Accordingly, theseries of tasks described above may be encapsulated within machine codecomprising the series of instructions. In some embodiments, as explainedbelow with respect to FIG. 8, the series of tasks may be defined by acompiler configured to distribute a higher-level series of tasks amongstthe plurality of logic circuits as a plurality of series of tasks. Forexample, the compiler may generate the plurality of series of tasksbased on the higher-level series of tasks such that the processorsubunits, executing each corresponding series of tasks in tandem,perform the same function as outlined by the higher-level series oftasks.

As explained further below, the higher-level series of tasks maycomprise a set of instructions in a human-readable programming language.Correspondingly, the series of tasks for each processor subunit maycomprise lower-level series of tasks, each of which comprises a set ofinstructions in a machine code.

As explained above with respect to FIGS. 7A and 7B, the memory chip mayfurther comprise a plurality of buses, each bus connecting one of theplurality of processor subunits to at least one other of the pluralityof processor subunits. Moreover, as explained above, data transfers onthe plurality of buses may be controlled using software. Accordingly,data transfers across at least one of the plurality of buses may bepredefined by the series of instructions included in a processor subunitconnected to the at least one of the plurality of buses. Therefore, oneof the tasks included in the series of instructions may includeoutputting data to one of the buses or pulling data from one of thebuses. Such tasks may be executed by a processing element of theprocessor subunit or by one or more accelerators included in theprocessor subunit. In the latter embodiment, the processor subunit mayperform a calculation or send a control signal to a corresponding memorybank in the same cycle during which accelerator(s) pull data from orplace data on one of the buses.

In one example, the series of instructions included in the processorsubunit connected to the at least one of the plurality of buses mayinclude a sending task that comprises a command for the processorsubunit connected to the at least one of the plurality of buses to writedata to the at least one of the plurality of buses. Additionally oralternatively, the series of instructions included in the processorsubunit connected to the at least one of the plurality of buses mayinclude a receiving task that comprises a command for the processorsubunit connected to the at least one of the plurality of buses to readdata from the at least one of the plurality of buses.

Additionally or alternatively to distribution of code amongst processorsubunits, data may be divided between memory banks of the memory chip.For example, as explained above, a distributed processor on a memorychip may comprise a plurality of processor subunits disposed on thememory chip and a plurality of memory banks disposed on the memory chip.Each one of the plurality of memory banks may be configured to storedata independent from data stored in other ones of the plurality ofmemory banks, and each one of the plurality of processor subunits may beconnected to at least one dedicated memory bank from among the pluralityof memory banks. For example, each processor subunit may have access toone or more memory controllers of one or more corresponding memory banksdedicated to the processor subunit, and no other processor subunit mayhave access to these corresponding one or more memory controllers.Accordingly, the data stored in each memory bank may be unique to thededicated processor subunit. Moreover, the data stored in each memorybank may be independent of the memory stored in other memory banksbecause no memory controllers may be shared between memory banks.

In some embodiments, as described below with respect to FIG. 8, the datastored in each of the plurality of memory banks may be defined by acompiler configured to distribute data amongst the plurality of memorybanks. Moreover, the compiler may be configured to distribute datadefined in a higher-level series of tasks amongst the plurality ofmemory banks using a plurality of lower-level tasks distributed amongstcorresponding processor subunits.

As explained further below, the higher-level series of tasks maycomprise a set of instructions in a human-readable programming language.Correspondingly, the series of tasks for each processor subunit maycomprise lower-level series of tasks, each of which comprises a set ofinstructions in a machine code.

As explained above with respect to FIGS. 7A and 7B, the memory chip mayfurther comprise a plurality of buses, each bus connecting one of theplurality of processor subunits to one or more corresponding, dedicatedmemory banks from among the plurality of memory banks. Moreover, asexplained above, data transfers on the plurality of buses may becontrolled using software. Accordingly, data transfers across aparticular one of the plurality of buses may be controlled by acorresponding processor subunit connected to the particular one of theplurality of buses. Therefore, one of the tasks included in the seriesof instructions may include outputting data to one of the buses orpulling data from one of the buses. As explained above, such tasks maybe executed by (i) a processing element of the processor subunit or (ii)one or more accelerators included in the processor subunit. In thelatter embodiment, the processor subunit may perform a calculation oruse buses connecting the processor subunit to other processor subunitsin the same cycle during which accelerator(s) pull data from or placedata on one of the buses connected to the one or more corresponding,dedicated memory banks.

Therefore, in one example, the series of instructions included in theprocessor subunit connected to the at least one of the plurality ofbuses may include a sending task. The sending task may comprise acommand for the processor subunit connected to the at least one of theplurality of buses to write data to the at least one of the plurality ofbuses for storage in the one or more corresponding, dedicated memorybanks Additionally or alternatively, the series of instructions includedin the processor subunit connected to the at least one of the pluralityof buses may include a receiving task. The receiving task may comprise acommand for the processor subunit connected to the at least one of theplurality of buses to read data from the at least one of the pluralityof buses for storage in the one or more corresponding, dedicated memorybanks. Accordingly, the sending and receiving tasks in such embodimentsmay comprise control signals that are sent, along the at least one ofthe plurality of buses, to one or more memory controllers of the one ormore corresponding, dedicated memory banks. Moreover, the sending andreceiving tasks may be executed by one portion of the processing subunit(e.g., by one or more accelerators thereof) concurrently with acalculation or other task executed by another portion of the processingsubunit (e.g., by one or more different accelerators thereof). Anexample of such a concurrent execution may include a MAC-relay command,in which receiving, multiplying, and sending are executed in tandem.

In addition to distributing data amongst the memory banks, particularportions of data may be duplicated across different memory banks. Forexample, as explained above, a distributed processor on a memory chipmay comprise a plurality of processor subunits disposed on the memorychip and a plurality of memory banks disposed on the memory chip. Eachone of the plurality of processor subunits may be connected to at leastone dedicated memory bank from among the plurality of memory banks, andeach memory bank of the plurality of memory banks may be configured tostore data independent from data stored in other ones of the pluralityof memory banks. Moreover, at least some of the data stored in oneparticular memory bank from among the plurality of memory banks maycomprise a duplicate of data stored in at least another one of theplurality of memory banks. For example, a number, string, or other typeof data used in the series of instructions may be stored in a pluralityof memory banks dedicated to different processor subunits rather thanbeing transferred from one memory bank to other processor subunits inthe memory chip.

In one example, parallel string matching may use data duplicationdescribed above. For example, a plurality of strings may be compared tothe same string. A conventional processor would compare each string inthe plurality to the same string in sequence. On a hardware chip of thepresent disclosure, the same string may be duplicated across the memorybanks such that the processor subunits may compare a separate string inthe plurality to the duplicated string in parallel.

In some embodiments, as described below with respect to FIG. 8, the atleast some data duplicated across the one particular memory bank fromamong the plurality of memory banks and the at least another one of theplurality of memory banks is defined by a compiler configured toduplicate data across memory banks. Moreover, the compiler may beconfigured to duplicate the at least some data using a plurality oflower-level tasks distributed amongst corresponding processor subunits.

Duplication of data may be useful for certain tasks that re-use the sameportions of data across different calculations. By duplicating theseportions of data, the different calculations may be distributed amongstprocessor subunits of the memory chip for parallel execution while eachprocessor subunit may store the portions of data in, and access thestored portions from, a dedicated memory bank (rather than pushing andpulling the portions of data across buses connecting the processorsubunits). In one example, the at least some data duplicated across theone particular memory bank from among the plurality of memory banks andthe at least another one of the plurality of memory banks may compriseweights of a neural network. In this example, each node in the neuralnetwork may be defined by at least one processor subunit from among theplurality of processor subunits. For example, each node may comprisemachine code executed by the at least one processor subunit defining thenode. In this example, duplication of the weights may allow eachprocessor subunit to execute machine code to effect, at least in part, acorresponding node while only accessing one or more dedicated memorybanks (rather than performing data transfers with other processorsubunits). Because the timing of reads and writes to the dedicatedmemory bank(s) are independent of other processor subunits while thetiming of data transfers between processor subunits requires timingsynchronization (e.g., using software, as explained above), duplicationof memory to avoid data transfers between processor subunits may producefurther efficiencies in overall execution.

As explained above with respect to FIGS. 7A and 7B, the memory chip mayfurther comprise a plurality of buses, each bus connecting one of theplurality of processor subunits to one or more corresponding, dedicatedmemory banks from among the plurality of memory banks. Moreover, asexplained above, data transfers on the plurality of buses may becontrolled using software. Accordingly, data transfers across aparticular one of the plurality of buses may be controlled by acorresponding processor subunit connected to the particular one of theplurality of buses. Therefore, one of the tasks included in the seriesof instructions may include outputting data to one of the buses orpulling data from one of the buses. As explained above, such tasks maybe executed by (i) a processing element of the processor subunit or (ii)one or more accelerators included in the processor subunit. As furtherexplained above, such tasks may include a sending task and/or areceiving tasks that comprise control signals that are sent, along theat least one of the plurality of buses, to one or more memorycontrollers of the one or more corresponding, dedicated memory banks.

FIG. 8 depicts a flowchart of a method 800 for compiling a series ofinstructions for execution on an exemplary memory chip of the presentdisclosure, e.g., as depicted in FIGS. 7A and 7B. Method 800 may beimplemented by any conventional processor, whether generic orspecial-purpose.

Method 800 may be executed as a portion of a computer program forming acompiler. As used herein, a “compiler” refers to any computer programthat converts a higher-level language (e.g., a procedural language, suchas C, FORTRAN, BASIC, or the like; an object-oriented language, such asJava, C++, Pascal, Python, or the like; etc.) to a lower-level language(e.g., assembly code, object code, machine code, or the like). Thecompiler may allow a human to program a series of instructions in ahuman-readable language, which is then converted to a machine-executablelanguage.

At step 810, the processor may assign tasks associated with the seriesof instructions to different ones of the processor subunits. Forexample, the series of instructions may be divided into subgroups, thesubgroups to be executed in parallel across the processor subunits. Inone example, a neural network may be divided into its nodes, and one ormore nodes may be assigned to separate processor subunits. In thisexample, each subgroup may comprise a plurality of nodes connectedacross different layers. Thus, a processor subunit may implement a nodefrom a first layer of the neural network, a node from a second layerconnected to the node from the first layer implemented by the sameprocessor subunit, and the like. By assigning nodes based on theirconnections, data transfers between the processor subunits may belessened, which may result in greater efficiency, as explained above.

As explained above depicted in FIGS. 7A and 7B, the processor subunitsmay be spatially distributed among the plurality of memory banksdisposed on the memory chip. Accordingly, the assignment of tasks maybe, at least in part, a spatial divisional as well as a logicaldivision.

At step 820, the processor may generate tasks to transfer data betweenpairs of the processor subunits of the memory chip, each pair ofprocessor subunits being connected by a bus. For example, as explainedabove, the data transfers may be controlled using software. Accordingly,processor subunits may be configured to push and pull data on buses atsynchronized times. The generated tasks may thus include tasks forperforming this synchronized pushing and pulling of data.

As explained above, step 820 may include pre-processing to account forthe internal behavior, including timing and latencies, of the processorsubunits. For example, the processor may use known times and latenciesof the processor subunits (e.g., the time to push data to a bus, thetime to pull data from a bus, the latency between a calculation and apush or pull, or the like) to ensure that the generated taskssynchronize. Therefore, the data transfers comprising at least one pushby one or more processor subunits and at least one pull by one or moreprocessor subunits may occur simultaneously rather than incurring adelay due to timing differences between the processor subunits,latencies of the processor subunits, or the like.

At step 830, the processor may group the assigned and generated tasksinto the plurality of groups of sub-series instructions. For example,the sub-series instructions may each comprise a series of tasks forexecution by a single processor subunit. Therefore, each of theplurality of groups of sub-series instructions may correspond to adifferent one of the plurality of processor sub-units. Accordingly,steps 810, 820, and 830 may result in dividing the series ofinstructions into a plurality of groups of sub-series instructions. Asexplained above, step 820 may ensure that any data transfers between thedifferent groups are synchronized.

At step 840, the processor may generate machine code corresponding toeach of the plurality of groups of subs-series instructions. Forexample, the higher-level code representing sub-series instructions maybe converted to lower-level code, such as machine code, executable bycorresponding processor subunits.

At step 850, the processor may assign the generated machine codecorresponding to each of the plurality of groups of subs-seriesinstructions to a corresponding one of the plurality of processorsubunits in accordance with the division. For example, the processor maylabel each sub-series instructions with an identifier of thecorresponding processor subunit. Thus, when the sub-series instructionsare uploaded to a memory chip for execution (e.g., by host 350 of FIG.3A), each sub-series may configure a correct processor subunit.

In some embodiments, assigning tasks associated with the series ofinstructions to the different ones of the processor subunits may depend,at least in part, on a spatial proximity between two or more of theprocessor subunits on the memory chip. For example, as explained above,efficiency may be increased by lessening the number of data transfersbetween processor subunits. Accordingly, the processor may minimize datatransfers that move data across more than two of processor subunits.Therefore, the processor may use a known layout of the memory chip incombination with one or more optimization algorithms (such as a greedyalgorithm) in order to assign sub-series to processor subunits in a waythat maximizes (at least locally) adjacent transfers and minimizes (atleast locally) transfers to non-neighboring processor subunits.

Method 800 may include further optimizations for the memory chips of thepresent disclosure. For example, the processor may group data associatedwith the series of instructions based on the division and assign thedata to the memory banks in accordance with the grouping. Accordingly,the memory banks may hold data used for the sub-series instructionsassigned to each processor subunit to which each memory bank isdedicated.

In some embodiments, grouping the data may include determining at leasta portion of the data to duplicate in two or more of the memory banks.For example, as explained above, some data may be used across more thanone sub-series instructions. Such data may be duplicated across thememory banks dedicated to the plurality of processor subunits to whichthe different sub-series instructions are assigned. This optimizationmay further reduce data transfers across processor subunits.

The output of method 800 may be input to a memory chip of the presentdisclosure for execution. For example, a memory chip may comprise aplurality of processor subunits and a corresponding plurality of memorybanks, each processor subunit being connected to at least one memorybank dedicated to the processor subunit, and the processor subunits ofthe memory chip may be configured to execute the machine code generatedby method 800. As explained above with respect to FIG. 3A, host 350 mayinput the machine code generated by method 800 to the processor subunitsfor execution.

Sub-Banks and Sub-Controllers

In conventional memory banks, controllers are provided at the banklevel. Each bank includes a plurality of mats, which are typicallyarranged in a rectangular manner but may be arranged in any geometricalshape. Each mat includes a plurality of memory cells, which are alsotypically arranged in a rectangular manner but may be arranged in anygeometrical shape. Each cell may store a single bit of data (e.g.,depending on whether the cell is retained at a high voltage or a lowvoltage).

An example of this conventional architecture is depicted in FIGS. 9 and10. As shown in FIG. 9, at the bank level, a plurality of mats (e.g.,mats 930-1, 930-2, 940-1, and 940-2) may form bank 900. In aconventional rectangular organization, bank 900 may be controlled acrossglobal wordlines (e.g., wordline 950) and global bitlines (e.g., bitline960). Accordingly, row decoder 910 may select the correct wordline basedon an incoming control signal (e.g., a request for a read from anaddress, a request for a write to an address, or the like) and globalsense amplifier 920 (and/or a global column decoder, not shown in FIG.9) may select the correct bitline based on the control signal. Amplifier920 may also amplify any voltage levels from a selected bank during aread operation. Although depicted as using a row decoder for initialselecting and performing amplification along columns, a bank mayadditionally or alternatively use a column decoder for initial selectingand perform amplification along rows.

FIG. 10 depicts an example of a mat 1000. For example, mat 1000 may forma portion of a memory bank, such as bank 900 of FIG. 9. As depicted inFIG. 10, a plurality of cells (e.g., cells 1030-1, 1030-2, and 1030-3)may form mat 1000. Each cell may comprise a capacitor, a transistor, orother circuitry that stores at least one bit of data. For example, acell may comprise a capacitor that is charged to represent a ‘1’ anddischarged to represent a ‘0’ or may comprise a flip-flop having a firststate representing a ‘1’ and a second state representing a ‘0.’ Aconventional mat may comprise, for example, 512 bits by 512 bits. Inembodiments where mat 1000 forms a portion of MRAM, ReRAM, or the like,a cell may comprise a transistor, resistor, capacitor or other mechanismfor isolating an ion or portion of a material that stores at least onebit of data. For example, a cell may comprise a electrolyte ion, aportion of chalcogenide glass, or the like, having a first staterepresenting a ‘1’ and a second state representing a ‘0.’

As further depicted in FIG. 10, in a conventional rectangularorganization, mat 1000 may be controlled across local wordlines (e.g.,wordline 1040) and local bitlines (e.g., bitline 1050). Accordingly,wordline drivers (e.g., wordline driver 1020-1, 1020-2, . . . , 1020-x)may control the selected wordline to perform a read, write, or refreshbased on a control signal from a controller associated with the memorybank of which mat 1000 forms a part (e.g., a request for a read from anaddress, a request for a write to an address, a refresh signal).Moreover, local sense amplifiers (e.g., local amplifiers 1010-1, 1010-2,. . . , 1010-x) and/or local column decoders (not shown in FIG. 10) maycontrol the selected bitline to perform a read, write, or refresh. Thelocal sense amplifiers may also amplify any voltage levels from aselected cell during a read operation. Although depicted as using awordline driver for initial selecting and performing amplification alongcolumns, a mat may instead use a bitline driver for initial selectingand perform amplification along rows.

As explained above, a large number of mats are duplicated to form amemory bank. Memory banks may be grouped to form a memory chip. Forexample, a memory chip may comprise eight to thirty-two memory banks.Accordingly, pairing processor subunits with memory banks on aconventional memory chip may result in only eight to thirty twoprocessor subunits. Accordingly, embodiments of the present disclosuremay include memory chips with additional sub-bank hierarchy. Thesememory chips of the present disclosure may then include processorsubunits with memory sub-banks used as the dedicated memory banks pairedwith the processor subunits allowing for a larger number of subprocessors, which may then achieve higher parallelism and performance ofin-memory computing.

In some embodiments of the present disclosure, the global row decoderand global sense amplifier of bank 900 may be replaced with sub-bankcontrollers. Accordingly, rather than sending control signals to aglobal row decoder and a global sense amplifier of the memory bank, acontroller of the memory bank may direct the control signal to theappropriate sub-bank controller. The direction may be controlleddynamically or may be hard-wired (e.g., via one or more logic gates). Insome embodiments, fuses may be used to indicate the controller of eachsub bank or mat whether to block or pass the control signal to theappropriate sub-bank or mat. In such embodiments, faulty sub-banks maythus be deactivated using the fuses.

In one example of such embodiments, a memory chip may include aplurality of memory banks, each memory bank having a bank controller anda plurality of memory sub-banks, each memory sub-bank having a sub-bankrow decoder and a sub-bank column decoder for allowing reads and writesto locations on the memory sub-bank. Each sub-bank may comprise aplurality of memory mats, each memory mat having a plurality of memorycells and may have internally local row decoders, column decoders,and/or local sense amplifiers. The sub-bank row decoders and thesub-bank column decoders may process read and write requests from thebank controller or from a sub-bank processor subunit used for in memorycomputations on the sub-bank memory, as described below. Additionally,each memory sub-bank may further have a controller configured todetermine whether to process read requests and write requests from thebank controller and/or to forward them to the next level (e.g., of rowand column decoders on a mat) or to block the requests, e.g., to allowan internal processing element or processor subunit to access thememory. In some embodiments, the bank controller may be synchronized toa system clock. However, the sub-bank controllers may be notsynchronized to the system clock.

As explained above, the use of sub-banks may allow for the inclusion ofa larger number processor subunits in the memory chip than if processorsubunits were paired with memory banks of conventional chips.Accordingly, each sub-bank may further have a processor subunit usingthe sub-bank as a dedicated memory. As explained above, the processorsubunit may comprise a RISC, a CISC, or other general-purpose processingsubunit and/or may comprise one or more accelerators. Additionally, theprocessor subunit may include an address generator, as explained above.In any of the embodiments described above, each processor subunit may beconfigured to access a sub-bank dedicated to the processor subunit usingthe row decoder and the column decoder of the sub-bank without using thebank controller. The processor sub-unit associated with the sub-bank mayalso handle the memory mats (including the decoder and memory redundancymechanisms, described below) and/or determine whether a read or writerequest from an upper level (e.g., the bank level or the memory level)is forwarded and handled accordingly.

In some embodiments, the sub-bank controller may further include aregister that stores a state of the sub-bank. Accordingly, the sub-bankcontroller may return an error if the sub-bank controller receives acontrol signal from the memory controller while the register indicatesthat the sub-bank is in use. In embodiments where each sub-bank furtherincludes a processor subunit, the register may indicate an error if theprocessor subunit in the sub-bank is accessing the memory in conflictwith an external request from the memory controller.

FIG. 11 shows an example of another embodiment of a memory bank usingsub-bank controllers. In the example of FIG. 11, bank 1100 has a rowdecoder 1110, a column decoder 1120, and a plurality of memory sub-banks(e.g., sub-banks 1170 a, 1170 b, and 1170 c) with sub-bank controllers(e.g., controllers 1130 a, 1130 b, and 1130 c). The sub-bank controllersmay include address resolvers (e.g., resolvers 1140 a, 1140 b, and 1140c), which may determine whether to pass a request to one or moresub-banks controlled by the sub-bank controller.

The sub-bank controllers may further include one or more logic circuits(e.g., logic 1150 a, 1150 b, and 1150 c). For example, a logic circuitcomprising one or more processing elements may allow for one or moreoperations, such as refreshing of cells in the sub-bank, clearing ofcells in the sub-bank, or the like, to be performed without processingrequests externally from bank 1100. Alternatively, the logic circuit maycomprise a processor subunit, as explained above, such that theprocessor sub-unit has any sub-banks controlled by the sub-bankcontroller as corresponding, dedicated memory. In the example of FIG.11, logic 1150 a may have sub-bank 1170 a as a corresponding, dedicatedmemory, logic 1150 b may have sub-bank 1170 b as a corresponding,dedicated memory, and logic 1150 c may have sub-bank 1170 c as acorresponding, dedicated memory. In any of the embodiments describedabove, the logic circuits may have buses to the sub-banks, e.g., buses1131 a, 1131 b, or 1131 c. As further depicted in FIG. 11, the sub-bankcontrollers may each include a plurality of decoders, such as a sub-bankrow decoder and a sub-bank column decoder for allowing reads and writes,either by a processing element or processor subunit or by a higher-levelmemory controller issuing commands, to locations on the memorysub-bank(s). For example, sub-bank controller 1130 a includes decoders1160 a, 1160 b, and 1160 c, sub-bank controller 1130 b includes decoders1160 d, 1160 e, and 1160 f, and sub-bank controller 1130 c includesdecoders 1160 g, 1160 h, and 1160 i. The sub-bank controllers may, basedon a request from bank row decoder 1110, select a wordline using thedecoders included in the sub-bank controllers. The described system mayallow a processing element or processor subunit of the sub-bank toaccess the memory without interrupting other banks and even othersub-banks, thereby allowing each sub-bank processor subunit to performmemory computations in parallel with the other sub-bank processorsubunits.

Furthermore, each sub-bank may comprise a plurality of memory mats, eachmemory mat having a plurality of memory cells. For example, sub-bank1170 a includes mats 1190 a-1, 1190 a-2, . . . , 1190 a-x; sub-bank 1170b includes mats 1190 b-1, 1190 b-2, . . . , 1190 b-x; and sub-bank 1170c includes mats 1190 c-1, 1190 c-2, . . . , 1190 c-3. As furtherdepicted in FIG. 11, each sub-bank may include at least one decoder. Forexample, sub-bank 1170 a includes decoder 1180 a, sub-bank 1170 bincludes decoder 1180 b, and sub-bank 1170 c includes decoder 1180 c.Accordingly, bank column decoder 1120 may select a global bitline (e.g.,bitline 1121 a or 1121 b) based on external requests while the sub-bankselected by bank row decoder 1110 may use its column decoder to select alocal bitline (e.g., bitline 1181 a or 1181 b) based on local requestsfrom the logic circuit to which the sub-bank is dedicated. Accordingly,each processor subunit may be configured to access a sub-bank dedicatedto the processor subunit using the row decoder and the column decoder ofthe sub-bank without using the bank row decoder and the bank columndecoder. Thus, each processor subunit may access a correspondingsub-bank without interrupting other sub-banks. Moreover, sub-bankdecoders may reflect accessed data to the bank decoders when the requestto the sub-bank is external to the processor subunit. Alternatively, inembodiments where each sub-bank has only one row of memory mats, thelocal bitlines may be the bitlines of the mat rather than bitlines ofthe sub-bank.

A combination of embodiments using sub-bank row decoders and sub-bankcolumn decoders with the embodiment depicted in FIG. 11 may be used. Forexample, the bank row decoder may be eliminated but the bank columndecoder retained and local bitlines used.

FIG. 12 shows an example of an embodiment of a memory sub-bank 1200having a plurality of mats. For example, sub-bank 1200 may represent aportion of sub-bank 1100 of FIG. 11 or may represent an alternativeimplementation of a memory bank. In the example of FIG. 12, sub-bank1200 includes a plurality of mats (e.g., mats 1240 a and 1240 b).Moreover, each mat may include a plurality of cells. For example, mat1240 a includes cells 1260 a-1, 1260 a-2, . . . , 1260 a-x, and mat 1240b includes cells 1260 b-1, 1260 b-2, . . . , 1260 b-x.

Each mat may be assigned a range of addresses that will be assigned tothe memory cells of the mat. These addresses may be configured atproduction such that mats may be shuffled around and such that faultedmats may be deactivated and left unused (e.g., using one or more fuses,as explained further below).

Sub-bank 1200 receives read and write requests from memory controller1210. Although not depicted in FIG. 12, requests from memory controller1210 may be filtered through a controller of sub-bank 1200 and directedto an appropriate mat of sub-bank 1200 for address resolution.Alternatively, at least a portion (e.g., higher bits) of an address of arequest from memory controller 1210 may be transmitted to all mats ofsub-bank 1200 (e.g., mats 1240 a and 1240 b) such that each mat mayprocess the full address and the request associated with the addressonly if the mat's assigned address range includes the address specifiedin the command Similar to the sub-bank direction described above, themat determination may be dynamically controlled or may be hardwired. Insome embodiments, fuses may be used to determine the address range foreach mat, also allowing for disabling of faulty mats by assigning anillegal address range. Mats may additionally or alternatively bedisabled by other common methods or connection of fuses.

In any of the embodiments described above, each mat of the sub-bank mayinclude a row decoder (e.g., row decoder 1230 a or 1230 b) for selectionof a wordline in the mat. In some embodiments, each mat may furtherinclude fuses and comparators (e.g., 1220 a and 1220 b). As describedabove, the comparators may allow each mat to determine whether toprocess an incoming request, and the fuses may allow each mat todeactivate if faulty. Alternatively, row decoders for the bank and/orsub-bank may be used rather than a row decoder in each mat.

Furthermore, in any of the embodiments described above, a column decoderincluded in the appropriate mat (e.g., column decoder 1250 a or 1250 b)may select a local bitline (e.g., bitline 1251 or 1253). The localbitline may be connected to a global bitline of the memory bank. Inembodiments where the sub-bank has local bitlines of its own, the localbitline of the cell may be further connected to the local bitline of thesub-bank. Accordingly, data in the selected cell may be read through thecolumn decoder (and/or sense amplifier) of the cell, then through thecolumn decoder (and/or sense amplifier) of the sub-bank (in embodimentsincluding a sub-bank column decoder and/or sense amplifier), and thenthrough the column decoder (and/or sense amplifier) of the bank.

Mat 1200 may be duplicated and arrayed to form a memory bank (or amemory sub-bank). For example, a memory chip of the present disclosuremay comprise a plurality of memory banks, each memory bank having aplurality of memory sub-banks, and each memory sub-bank having asub-bank controller for processing reads and writes to locations on thememory sub-bank. Furthermore, each memory sub-bank may comprise aplurality of memory mats, each memory mat having a plurality of memorycells and having a mat row decoder and a mat column decoder (e.g., asdepicted in FIG. 12). The mat row decoders and the mat column decodersmay process read and write requests from the sub-bank controller. Forexample, the mat decoders may receive all requests and determine (e.g.,using a comparator) whether to process the request based on a knownaddress range of each mat, or the mat decoders may only receive requestswithin the known address range based on selection of a mat by thesub-bank (or bank) controller.

Controller Data Transfers

Any of the memory chips of the present disclosure may also share datausing memory controllers (or sub-bank controllers or mat controllers) inaddition to sharing data using processing subunits. For example, amemory chip of the present disclosure may comprise a plurality of memorybanks (e.g., an SRAM bank, a DRAM bank, or the like), each memory bankhaving a bank controller, a row decoder, and a column decoder forallowing reads and writes to locations on the memory bank, as well as aplurality of buses connecting each controller of the plurality of bankcontrollers to at least one other controller of the plurality of bankcontrollers. The plurality of buses may be similar to the busesconnecting the processing subunits, as described above, but connectingthe bank controllers directly rather than through the processingsubunits. Furthermore, although described as connecting the bankcontrollers, buses may additionally or alternatively connect sub-bankcontrollers and/or mat controllers.

In some embodiments, the plurality of buses may be accessed withoutinterruption of data transfers on main buses of the memory banksconnected to one or more processor subunits. Accordingly, a memory bank(or sub-bank) may transmit data to or from a corresponding processorsubunit in the same clock cycle as transmitting data to or from adifferent memory bank (or sub-bank). In embodiments where eachcontroller is connected to a plurality of other controllers, thecontrollers may be configurable for selection of one other of the othercontrollers for sending or receiving of data. In some embodiments, eachcontroller may be connected to at least one neighboring controller(e.g., pairs of spatially adjacent controllers may be connected to oneanother).

Redundant Logic in Memory Circuits

The disclosure is generally directed to a memory chip with primary logicportions for on-chip data processing. The memory chip may includeredundant logic portions, which may replace defective primary logicportions to increase the fabrication yield of the chip. Thus, the chipmay include on-chip components that allow a configuration of logicblocks in the memory chip based on individual testing of the logicportions. This feature of the chip may increase yields because a memorychip with larger areas dedicated to logic portions is more susceptibleto fabrication failures. For example, DRAM memory chips with largeredundant logic portions may be susceptible to fabrication issues thatreduce yield. However, implementing redundant logic portions may resultin increased yield and reliability because it provides a manufacturer oruser of DRAM memory chips to turn on or off full logic portions whilemaintaining the ability of high parallelism. It should be noted thathere and throughout the disclosure, example of certain memory types(such as DRAM) may be identified in order to facilitate the explanationof disclosed embodiments. It is to be understood, however, that in suchinstances the identified memory types are not intended to be limiting.Rather, memory types such as DRAM, Flash, SRAM, ReRAM, PRAM, MRAM, ROM,or any other memory may be used together with the disclosed embodimentseven if fewer examples are specifically identified in a certain sectionof the disclosure.

FIG. 13 is a block diagram of an exemplary memory chip 1300, consistentwith disclosed embodiments. Memory chip 1300 may be implemented as aDRAM memory chip. Memory chip 1300 may also be implemented as any typeof memory volatile or non-volatile, such as Flash, SRAM, ReRAM, PRAM,and/or MRAM, etc. Memory chip 1300 may include a substrate 1301 in whichan address manager 1302, a memory array 1304 including a plurality ofmemory banks, 1304(a,a) to 1304(z,z), a memory logic 1306, a businesslogic 1308, and a redundant business logic 1310 are disposed. Memorylogic 1306 and business logic 1308 may constitute primary logic blocks,while redundant business logic 1310 may constitute redundant blocks. Inaddition, memory chip 1300 may include configuration switches, which mayinclude deactivation switches 1312, and an activation switches 1314.Deactivation switches 1312 and activation switches 1314 may also bedisposed in the substrate 1301. In this Application, memory logic 1306,business logic 1308, and redundant business logic 1310 may also becollectively referred to as the “logic blocks.”

Address manager 1302 may include row and column decoders or other typeof memory auxiliaries. Alternatively, or additionally, address manager1302 may include a microcontroller or processing unit.

In some embodiments, as shown in FIG. 13, memory chip 1300 may include asingle memory array 1304 that may arrange the plurality of memory blocksin a two-dimensional array on substrate 1301. In other embodiments,however, memory chip 1300 may include multiple memory arrays 1304 andeach of the memory arrays 1304 may arrange memory blocks in differentconfigurations. For example, memory blocks in at least one of the memoryarrays (also known as memory banks) may be arranged in a radialdistribution to facilitate routing between address manager 1302 ormemory logic 1306 to the memory blocks.

Business logic 1308 may be used to do the in-memory computation of anapplication that is not related to the logic used to manage the memoryitself. For example, business logic 1308 may implement functions relatedto AI such as floating, integer, or MAC operations used as activationfunctions. In addition, business logic 1308 may implement data baserelated functions like min, max, sort, count, among others. Memory logic1306 may perform tasks related to memory management, including (but notlimited to) read, write, and refresh operations. Therefore, businesslogic may be added in one or more of the bank level, mats level, or agroup of mats level. Business logic 1308 may have one or more addressoutputs and one or more data inputs/outputs. For instance, businesslogic 1308 can address by row\column lines to address manager 1302. Incertain embodiments, however, the logic blocks may be additionally oralternatively addressed via data inputs\outputs.

Redundant business logic 1310 may be a replicate of business logic 1308.In addition, redundant business logic 1310 may be connected todeactivation switches 1312 and/or activation switches 1314, which mayinclude small fuse\anti-fuse, and used for logic disabling or enablingone of the instances (e.g., an instance which is connected by default)and enable one of the other logic blocks (e.g., an instance which isdisconnected by default). In some embodiments, as further described inconnection to FIG. 15, the redundancy of blocks may be local within alogic block, such as business logic 1308.

In some embodiments, the logic blocks in memory chip 1300 may beconnected to subsets of memory array 1304 with dedicated buses. Forexample, a set of memory logic 1306, business logic 1308, and redundantbusiness logic 1310 may be connected to the first row of memory blocksin memory array 1304 (i.e., memory blocks 1304 (a,a) to 1304 (a,z)). Thededicated buses may allow associated logic blocks to quickly access datafrom the memory blocks without requirements of opening communicationlines through, for example, address manager 1302.

Each of the plurality of primary logic blocks may be connected to atleast one of the plurality of memory banks 1304. Also, redundant blocks,such as redundant business block 1310, may be connected to at least oneof the memory instances 1304(a,a)-(z,z). Redundant blocks may replicateat least one of the plurality of primary logic blocks, such as memorylogic 1306 or business logic 1308. Deactivation switches 1312 may beconnected to at least one of the plurality of primary logic blocks andactivation switches 1314 may be connected to at least one of theplurality of redundant blocks.

In these embodiments, upon detecting of a fault associated with one ofthe plurality of primary logic blocks (memory logic 1306 and/or businesslogic 1308), deactivation switches 1312 may be configured to disable theone of the plurality of primary logic blocks. Simultaneously, activationswitches 1314 may be configured to enable one of the plurality ofredundant blocks, such as redundant logic block 1310, that replicatesthe one of the plurality of primary logic blocks.

In addition, activation switches 1314 and deactivation switches 1312,which may collectively be referred to as “configuration switches,” mayinclude an external input to configure the status of the switch. Forinstance, activation switches 1314 may be configured so an activationsignal in the external input causes a closed switch condition, whiledeactivation switches 1312 may be configured so a deactivation signal inthe external input causes an open switch condition. In some embodiments,all configuration switches in 1300 may be deactivated by default andbecome activated or enabled after a test indicates an associated logicblock is functional and a signal is applied in the external input.Alternatively, in some cases, all configuration switches in 1300 may beenabled by default and may be deactivated or disabled after a testindicates an associated logic block is not functional and a deactivationsignal is applied in the external input.

Regardless of whether a configuration switch is initially enabled ordisabled, upon detection of a fault associated with an associated logicblock, the configuration switch may disable the associated logic block.In cases where the configuration switch is initially enabled, the stateof the configuration switch may be changed to disabled in order todisable the associated logic block. In cases where the configurationswitch is initially disabled, the state of the configuration switch maybe left in its disabled state in order to disable the associated logicblock. For example, the result of an operability test may indicate thata certain logic block is nonoperational or that it fails to operatewithin certain specifications. In such cases, the logic block may bedisabled my not enabling its corresponding configuration switch.

In some embodiments, configuration switches may be connected to two ormore logic blocks and may be configured to choose between differentlogic blocks. For example, a configuration switch may be connected toboth business logic 1308 and redundant logic block 1310. Configurationswitch may enable redundant logic block 1310 while disabling businesslogic 1308.

Alternatively, or additionally, at least one of the plurality of primarylogic blocks (memory logic 1306 and/or business logic 1308) may beconnected to a subset of the plurality of memory banks or memoryinstances 1304 with a first dedicated connection. Then, at least one ofthe plurality of redundant blocks (such as redundant business logic1310), which replicates the at least one of the plurality of primarylogic blocks, may be connected to the subset of the same plurality ofmemory banks or instances 1304 with a second dedicated connection.

Moreover, memory logic 1306 may have different functions andcapabilities than business logic 1308. For example, while memory logic1306 may be designed to enable read and write operations in the memorybank 1304, business logic 1308 may be designed to perform in-memorycomputations. Therefore, if the business logic 1308 includes a firstbusiness logic block, and the business logic 1308 includes a secondbusiness logic block (like redundant business logic 1310), it ispossible to disconnect defective business logic 1308 and reconnectredundant business logic 1310 without missing any capability.

In some embodiments, configuration switches (including deactivationswitches 1312 and activation switches 1314) may be implemented with afuse, an anti-fuse, or a programmable device (including a one-timeprogrammable device), or other form of non-volatile memory.

FIG. 14 is a block diagram of an exemplary redundant logic block set1400, consistent with disclosed embodiments. In some embodiments,redundant logic block set 1400 may be disposed in substrate 1301.Redundant logic block set 1400 may include at least one of businesslogic 1308, and redundant business logic 1310, connected to switches1312 and 1314, respectively. In addition, business logic 1308 andredundant business logic 1310 may be connected to an address bus 1402and a data bus 1404.

In some embodiments, as shown in FIG. 14, the switches 1312 and 1314 mayconnect logic blocks to a clock node. In this way, the configurationswitches may engage or disengage the logic blocks from the clock signal,effectively activating or deactivating the logic blocks. In otherembodiments, however, switches 1312 and 1314 may connect logic blocks toother nodes for activation or deactivation. For instance, configurationswitches may connect logic blocks to a voltage supply node (e.g., VCC)or to the ground node (e.g., GND) or clock signal. In this way, thelogic blocks may be enable of disable by the configuration switchesbecause they would create an open circuit or cut-off the logic blockpower supply.

In some embodiments, as shown in FIG. 14, address bus 1402 and data bus1404 may be in opposite sides of the logic blocks, which are connectedin parallel to each one of the buses. In this way, routing of thedifferent on-chip components may be facilitated by the logic block set1400.

In some embodiments, each one of the plurality of deactivation switches1312 couple at least one of the plurality of primary logic blocks with aclock node, and each one of the plurality of activation switches 1314may be couple at least one of the plurality of redundant blocks with theclock node allowing to connect\disconnect the clock as a simpleactivation\deactivation mechanism.

Redundant business logic 1310 of redundant logic block set 1400 allowsthe designer to choose, based on area and routing, the blocks that areworth duplication. For example, a chip designer may select larger blocksfor duplication because larger blocks may be more error prone. Thus, achip designer may decide to duplicate large logic blocks. On the otherhand, a designer may prefer to duplicate smaller logic blocks becausethey are easily duplicated without a significant loss of space.Moreover, using the configuration in FIG. 14, a designer may easilychoose to duplicate logic blocks depending on the statistics of errorsper area.

FIG. 15 is a block diagram for an exemplary logic block 1500, consistentwith disclosed embodiments. The logic block may be business logic 1308and/or redundant business logic 1310. In other embodiments, however, theexemplary logic block may describe memory logic 1306 or other componentof memory chip 1300.

Logic block 1500 presents yet another embodiment where the logicredundancy is used within a small processor pipeline. The logic block1500 may include a register 1508, a fetch circuit 1504, decoder 1506,and a write-back circuit 1518. In addition, logic block 1500 may includea computation unit 1510 and a duplicated computing unit 1512. However,in other embodiments, logic block 1500 may include other units that donot comprise a controller pipeline but include sporadic processingelements that comprise a required business logic.

Computation unit 1510 and duplicated computation unit 1512 may include adigital circuit capable of performing digital calculations. For example,computation unit 1510 and duplicated computation unit 1512 may includean arithmetic logic unit (ALU) to perform arithmetic and bitwiseoperations on binary numbers. Alternatively, computation unit 1510 andduplicated computation unit 1512 may include a floating-point unit(FPU), which operates on floating point numbers. In addition, in someembodiments computation unit 1510 and duplicated computation unit 1512may implement data base related functions like min, max, count, andcompare operations, among others.

In some embodiments, as shown in FIG. 15, computation unit 1510 andduplicated computation unit 1512 may be connected to switching circuits1514 and 1516. When activated the switching circuits may enable ordisable the computing units.

In logic block 1500, the duplicated computation unit 1512 may replicatethe computation unit 1510. Moreover, in some embodiments, register 1508,fetch circuit 1504, decoder 1506, and write-back circuit 1518(collectively referred to as the local logic units) may be smaller insize than the computation unit 1510. Because larger elements are moreprone to issues during fabrication, a designer may decide to replicatelarger units (such as computation unit 1510) instead of smaller units(such as the local logic units). Depending on historic yields and errorrates, however, a designed may elect to duplicate local logic unitsadditionally or alternatively to large units (or the entire block). Forexample, computation unit 1510 may be larger, and thus more error prone,than register 1508, fetch circuit 1504, decoder 1506, and write-backcircuit 1518. A designer may choose to duplicate computation unit 1510instead of the other elements in logic block 1500 or the whole block.

Logic block 1500 may include a plurality of local configurationswitches, each one of the plurality of local configuration switchesbeing connected to at least one of the at least one of computation unit1510 or duplicated computation unit 1512. Local configuration switchesmay be configured to disable computation unit 1510 and enable duplicatedcomputation unit 1512 when a fault is detected in the computation unit1510.

FIG. 16 shows block diagrams of exemplary logic blocks connected with abus, consistent with disclosed embodiments. In some embodiments, logicblocks 1602 (which may represent memory logic 1306, business logic 1308,or redundant business logic 1310) may be independent of each other, maybe connected via a bus, and may be activated externally by addressingthem specifically. For example, memory chip 1300 may include many logicblocks, each logic block having an ID number. In other embodiments,however, logic blocks 1602 may represent larger units comprised of aplurality one or more of memory logic 1306, business logic 1308, orredundant business logic 1310.

In some embodiments, each one of logic blocks 1602 may be redundant withthe other logic blocks 1602. This complete redundancy, in which allblocks may operate as primary or redundant blocks, may improvefabrication yields because a designer may disconnect faulty units whilemaintaining functionality of the overall chip. For example, a designermay have the ability to disable logic areas that are prone to errors butmaintain similar computation capabilities because the all duplicateblocks may be connected to the same address and data buses. For example,the initial number of logic blocks 1602 may greater than a targetcapability. Then, disabling some logic blocks 1602 would not affect thetarget capability.

A bus connected to the logic blocks may include address bus 1614,command lines 1616, and data lines 1618. As shown in FIG. 16, each oneof the logic blocks may be connected independently from each line in thebus. In certain embodiments, however, logic blocks 1602 may be connectedin a hierarchical structure to facilitate routing. For instance, eachline in the bus may be connected to a multiplexer that routes the lineto different logic blocks 1602.

In some embodiments, to allow external access without knowing theinternal chip structure, which may change due to enable and disabledunits, each one of the logic blocks may include Fused IDs such as fusedidentification 1604. Fused identification 1604 may include an array ofswitches (like fuses) that determine an ID and may be connected to amanaging circuit. For example, fused identification 1604 may beconnected to address manager 1302. Alternatively, fused identification1604 may be connected to higher memory address units. In theseembodiments, fused identification 1604 may be configurable to for aspecific address. For example, fused identification 1604 may include aprogrammable, non-volatile device that determines a final ID based oninstructions received form a managing circuit.

A distributed processor on a memory chip may be designed with theconfiguration depicted in FIG. 16. A testing procedure executed as BISTat chip wakeup or at factory testing may assign running ID numbers toblocks in the plurality of primary logic blocks (memory logic 1306 andbusiness logic 1308) that pass a testing protocol. A testing proceduremay also assign illegal ID numbers to blocks in the plurality of primarylogic blocks that do not pass the testing protocol. The test proceduremay also assign running ID numbers to blocks in the plurality ofredundant blocks (redundant logic block 1310) that pass the testingprotocol. Because redundant blocks replace failing primary logic blocks,the blocks in the plurality of redundant blocks assigned running IDnumbers may be equal to, or greater than, the blocks in the plurality ofprimary logic blocks assigned illegal ID numbers, thereby disabling theblock. In addition, each one of the plurality of primary logic blocksand each one of the plurality of redundant blocks may include at leastone fused identification 1604. Also, as shown in FIG. 16, the busconnecting logic blocks 1602 may include a command line, a data line,and an address line.

In other embodiments, however, all logic blocks 1602 that are connectedto the bus will start disabled and with no ID number. Tested one by one,each good logic block will get a running ID number, and those logicblocks not working will remain with illegal ID, which would disablethese blocks. In this manner, redundant logic blocks may improve thefabrication yields by replacing blocks that are known to be defectiveduring the testing process.

Address bus 1614 may couple a managing circuit to each one of theplurality of memory banks, each one of the plurality of primary logicblocks, and each one of the plurality of redundant blocks. Theseconnections allow the managing circuit to, upon detection of the faultassociated with a primary logic blocks (such as business logic 1308),assign an invalid address to the one of the plurality of primary logicblocks and assign a valid address to the one of the plurality ofredundant blocks.

For example, as shown in FIG. 16A, illegal IDs are configured to alllogic blocks 1602(a)-(c) (e.g., address 0xFFF). After testing logicblocks 1602(a) and 1602(c) are verified to be functional while logicblock 1602(b) is not functional. In FIG. 16A unshaded logic blocks mayrepresent logic blocks that passed the functionality test successfully,while shaded logic blocks may represent logic blocks that failed thetest for functionality. Then, the test procedure changes the illegal IDsto legal IDs for logic blocks that are functional while leaving theillegal IDs for logic blocks that are not functional. As an example, inFIG. 16A, the address for logic blocks 1602(a) and 1602(c) is changedfrom 0xFFF to 0x001 and 0x002, respectively. In contrast, the addressfor logic block 1602(b) remains the illegal address 0xFFF. In someembodiments, the ID is changed by programming a corresponding fusedidentification 1604.

Different results from the testing of logic blocks 1602 may result in adifferent configuration. For example, as shown in FIG. 16B, addressmanager 1302 may initially assign illegal IDs to all logic blocks 1602(i.e., 0xFFF). The testing results, however, may indicate that bothlogic blocks 1602(a) and 1602(b) are functional. In these cases, testingof logic block 1602(c) may not be necessary because memory chip 1300 mayrequire only two logic blocks. Therefore, to minimize testing resources,logic blocks may be tested only according to the minimum number offunctional logic blocks needed by the product definition of 1300,leaving other logic blocks untested. FIG. 16B also shows unshaded logicblocks, which represent tested logic blocks that passed the test forfunctionality, and shaded logic blocks, which represent untested logicblocks.

In these embodiments, a production tester (external or internal,automatic or manual) or a controller executing a BIST at startup, maychange illegal IDs to running IDs for tested logic blocks that arefunctional while leaving the illegal IDs to untested logic blocks. As anexample, in FIG. 16B, the address for logic blocks 1602(a) and 1602(b)is changed from 0xFFF to 0x001 and 0x002, respectively. In contrast, theaddress for untested logic block 1602(c) remains with the illegaladdress 0xFFF.

FIG. 17 is a block diagram for exemplary units 1702 and 1712 connectedin series, consistent with disclosed embodiments. FIG. 17 may representan entire system or chip. Alternatively, FIG. 17 may represent a blockin a chip containing other functional blocks.

Units 1702 and 1712 may represent complete units that include aplurality of logic blocks such as memory logic 1306 and/or businesslogic 1308. In these embodiments units 1702 and 1712 may also includeelements required to perform operations such as address manager 1302. Inother embodiments, however, units 1702 and 1712 may represent logicunits such as business logic 1308 or redundant business logic 1310.

FIG. 17 presents embodiments in which units 1702 and 1712 may need tocommunicate between themselves. In such cases, units 1702 and 1712 maybe connected in series. However, a non-working unit may break thecontinuity between the logic blocks. Therefore, the connection betweenunits may include a bypass option when a unit needs to be disabled dueto a defect. The bypass option can also be a part of the bypassed unititself.

In FIG. 17 units may be connected in series (e.g., 1702(a)-(c)), and afailing unit (e.g., 1702(b)) may be bypassed when it is defective. Theunits may further be connected in parallel with switching circuits. Forexample, in some embodiments units 1702 and 1712 may be connected withswitching circuits 1722 and 1728, as depicted in FIG. 17. In the exampledepicted in FIG. 17, unit 1702(b) is defective. For example, unit1702(b) does not pass a test for a circuit functionality. Therefore,unit 1702(b) may be disabled using, for example, activation switches1314 (not shown in FIG. 17) and/or switching circuit 1722(b) may beactivated to bypass unit 1702(b) and sustain the connectivity betweenlogic blocks.

Accordingly, when a plurality of primary units are connected in series,each one of the plurality of units may be connected in parallel with aparallel switch. Upon detection of a fault associated with the one ofthe plurality of units, the parallel switch connected to the one of theplurality of units may be activated to connect two of the plurality ofunits.

In other embodiments, as shown in FIG. 17, switching circuits 1728 mayinclude a sampling point or more that would cause a cycle or cyclesdelay maintaining synchronization between different lines of units. Whena unit is disabled, shorting the connection between adjacent logicblocks may generate synchronization errors with other calculations. Forexample, if a task requires data from both A and B lines, and each of Aand B is carried by an independent series of units, disabling a unitwould cause a desynchronization between the lines that would requirefurther data management. To prevent desynchronizations, sample circuits1730 may simulate the delay caused by the disabled unit 1712(b).Nonetheless, in some embodiments, the parallel switch may include ananti-fuse instead of a sampling circuit 1730.

FIG. 18 is a block diagram of exemplary units connected in atwo-dimension array, consistent with disclosed embodiments. FIG. 18 mayrepresent an entire system or chip. Alternatively, FIG. 18 may representa block in a chip containing other functional blocks.

Units 1806 may represent autonomous units that include a plurality oflogic blocks such as memory logic 1306 and/or business logic 1308.However, in other embodiments units 1806 may represent logic units suchas business logic 1308. Where convenient, discussion of FIG. 18 mayrefer to elements identified in FIG. 13 (e.g., memory chip 1300) anddiscussed above.

As shown in FIG. 18, units may be arranged in a two-dimensional array inwhich units 1806 (which may include or represent one or more of memorylogic 1306, business logic 1308, or redundant business logic 1310) areinterconnected via switching boxes 1808 and connection boxes 1810. Inaddition, in order to control the configuration of the two-dimensionalarray, the two-dimensional array may include I/O blocks 1804 in theperiphery of the two-dimensional array.

Connection boxes 1810 may be programmable and reconfigurable devicesthat may respond to signals inputted from the I/O blocks 1804. Forexample, connection boxes may include a plurality of input pins fromunits 1806 and may also be connected to switching boxes 1808.Alternatively, connection boxes 1810 may include a group of switchesconnecting pins of programmable logic cells with routing tracks, whileswitching boxes 1808 may include a group of switches connectingdifferent tracks.

In certain embodiments, connection boxes 1810 and switching boxes 1808may be implemented with configuration switches such as switches 1312 and1314. In such embodiments, connection boxes 1810 and switching boxes1808 may be configured by a production tester or a BIST executed at chipstartup.

In some embodiments, connection boxes 1810 and switching boxes 1808 maybe configured after units 1806 are tested for a circuit functionality.In such embodiments, I/O blocks 1804 may be used to send testing signalsto units 1806. Depending on the test results, I/O blocks 1804 may sendprogramming signals that configure connection boxes 1810 and switchingboxes 1808 in a manner disabling the units 1806 that fail the testingprotocol and enabling units 1806 that pass the testing protocol.

In such embodiments, the plurality of primary logic blocks and theplurality of redundant blocks may be disposed on the substrate in atwo-dimensional grid. Therefore, each one of the plurality of primaryunits 1806 and each one of the plurality of redundant blocks, such asredundant business logic 1310, may be interconnected with switchingboxes 1808, and an input block may be disposed in the periphery of eachline and each column of the two-dimensional grid.

FIG. 19 is a block diagram for exemplary units in a complex connection,consistent with disclosed embodiments. FIG. 19 may represent an entiresystem. Alternatively, FIG. 19 may represent a block in a chipcontaining other functional blocks.

The complex connection of FIG. 19 includes units 1902(a)-(f) andconfiguration switches 1904(a)-(h). Units 1902 may represent autonomousunits that include a plurality of logic blocks such as memory logic 1306and/or business logic 1308. However, in other embodiments units 1902 mayrepresent logic units such as memory logic 1306, business logic 1308, orredundant business logic 1310. Configuration switches 1904 may includeany of deactivation switches 1312 and activation switches 1314.

As shown in FIG. 19, the complex connection may include units 1902 intwo planes. For example, the complex connection may include twoindependent substrates separated in the z-axis. Alternatively, oradditionally, units 1902 may be arranged in two surfaces of a substrate.For example, with the objective to reduce the area of memory chip 1300,substrate 1301 may be arranged in two overlapping surfaces and connectedwith configuration switches 1904 arranged in three dimensions.Configuration switches may include deactivation switches 1312 and/oractivation switches 1314.

A first plane of the substrate may include “main” unit 1902. Theseblocks may be enabled by default. In such embodiments, a second plainmay include “redundant” unit 1902. These units may be disabled bydefault.

In some embodiments, configuration switches 1904 may include anti-fuses.Thus, after testing of units 1902, the blocks may be connected in a tileof functional units by switching certain anti-fuses to “always-on” anddisable selected units 1902, even if they are in a different plane. Inthe example presented in FIG. 19, one of the ‘main’ units (unit 1902(e))is not working. FIG. 19 may represent nonfunctional or untested blocksas shaded blocks while tested or functional blocks may be unshaded.Therefore, configuration switches 1904 are configured so one of thelogic blocks in a different plane (e.g., unit 1902(f) becomes active. Inthis way even though one of the main logic blocks was defective, thememory chip is still working by replacing a spare logic unit.

FIG. 19 additionally shows that one of the units 1902 (i.e., 1902(c)) inthe second plane is not tested or enabled because the main logic blocksare functional. For example, in FIG. 19, both main units 1902(a) and1902(d) passed a test for functionality. Thus, units 1902(c) was nottested or enabled. Therefore, FIG. 19 shows the ability to specificallyselect the logic blocks that become active depending on testing results.

In some embodiments, as shown in FIG. 19, not all units 1902 in a firstplain may have a corresponding spare or redundant blocks. However, inother embodiments, all units may be redundant with each other forcomplete redundancy where all units are both primary or redundant. Inaddition, while some implementations may follow the star networktopology depicted in FIG. 19, other implementation may use parallelconnections, serial connections, and/or couple the different elementswith configuration switches in parallel or in series.

FIG. 20 is an exemplary flowchart illustrating a redundant blockenabling process 2000, consistent with disclosed embodiments. Theenabling process 2000 may be implemented for memory chip 1300 andspecially for DRAM memory chips. In some embodiments, process 2000 mayinclude steps of testing each one of a plurality of logic blocks on thesubstrate of the memory chip for at least one circuit functionality,identifying faulty logic blocks in the plurality of primary logic blocksbased on the testing results, testing at least one redundant oradditional logic block on the substrate of the memory chip for the atleast one circuit functionality, disabling the at least one faulty logicblock by applying an external signal to a deactivation switch, andenabling the at least one redundant block by applying the externalsignal to an activation switch, the activation switch being connectedwith the at least one redundant block and being disposed on thesubstrate of the memory chip. The description of FIG. 20 below furtherelaborates on each step of process 2000.

Process 2000 may include testing a plurality of logic blocks (step2002), such as business block 1308 and a plurality of redundant blocks(e.g., redundant business block 1310). The testing may be beforepackaging using, for example, probing stations for on-wafer testing.Step 2000, however, may also be performed after packaging.

The testing in step 2002 may include applying a finite sequence oftesting signals to every logic block in memory chip 1300 or a subset oflogic blocks in memory chip 1300. The testing signals may includerequesting a computation that is expected to yield a 0 or a 1. In otherembodiments, the testing signal may request reading a specific addressin a memory bank or writing in a specific memory bank.

Testing techniques may be implemented to test the response of the logicblocks under iterative processes in step 2002. For example, the test mayinvolve testing logic blocks by transmitting instructions to write datain a memory bank and then verifying the integrity of the written data.In some embodiments, the testing may include repeating the algorithmwith data inversed.

In alternative embodiments, the testing of step 2002 may include runninga model of the logic blocks to generate a target memory image based on aset of testing instructions. Then, the same sequence of instructions maybe executed to the logic blocks in the memory chip, and the results maybe recorded. The residual memory image of the simulation may also becompared to the image taken from the test, and any mismatch may beflagged as a failure.

Alternatively, in step 2002, testing may include shadow modeling, wherea diagnostic is generated but the results are not necessarily predicted.Instead, the test using shadow modeling may be run in parallel on boththe memory chip and a simulation. For example, when the logic blocks inthe memory chip complete an instruction or task, the simulation may besignaled to execute the same instruction. Once the logic blocks in thememory chip finalize the instructions, the two models' architecturalstates may be compared. If there is a mismatch, then a failure isflagged.

In some embodiments, all logic blocks (including, e.g., each one ofmemory logic 1306, business logic 1308, or redundant business logic1310) may be tested in step 2002. In other embodiments, however, onlysubsets of the logic blocks may be tested in different testing rounds.For example, in a first round of testing only memory logic 1306 andassociated blocks may be tested. In a second round, only business logic1308 and associated blocks may be tested. In a third round, depending onthe results of the first two rounds, logic blocks associated withredundant business logic 1310 may be tested.

Process 2000 may continue to step 2004. In step 2004, faulty logicblocks may be identified, and faulty redundant blocks may also beidentified. For example, logic blocks that do not pass the testing ofstep 2002 may be identified as faulty blocks in step 2004. In otherembodiments, however, only certain faulty logic blocks may be initiallyidentified. For example, in some embodiments, only logic blocksassociated with business logic 1308 may be identified, and faultyredundant blocks are only identified if they are required forsubstituting a faulty logic block. In addition, identifying faultyblocks may include writing on a memory bank or a nonvolatile memory theidentification information of the identified faulty blocks.

In step 2006, faulty logic blocks may be disabled. For example, using aconfiguration circuit, the faulty logic blocks may be disabled bydisconnecting them from clock, ground, and/or power nodes.Alternatively, faulty logic blocks may be disabled by configuringconnection boxes in an arrangement that avoids the logic blocks. Yet, inother embodiments, faulty logic blocks may be disabled by receiving anillegal address from address manager 1302.

In step 2008, redundant blocks that duplicate the faulty logic blocksmay be identified. To support the same capabilities of the memory chipseven though some logic blocks have failed, in step 2008, redundantblocks that are available and can duplicate faulty logic blocks may beidentified. For example, if a logic block that performs multiplicationsof vectors is determined to be faulty, in step 2008, an address manager1302 or an on-chip controller may identify an available redundant logicblock that also performs multiplication of vectors.

In step 2010, the redundant blocks identified in step 2008 may beenabled. In contrast to the disable operation of step 2006, in step2010, the identified redundant blocks may be enabled by connecting themto clock, ground, and/or power nodes. Alternatively, identifiedredundant blocks may be enabled by configuring connection boxes in anarrangement that connects the identified redundant blocks. Yet, in otherembodiments, identified redundant blocks may be enabled by receiving arunning address at the test procedure execution time.

FIG. 21 is an exemplary flow chart illustrating an address assignmentprocess 2100, consistent with disclosed embodiments. The addressassignment process 2100 may be implemented for memory chip 1300 andspecially for a DRAM memory chips. As described in relation to FIG. 16,in some embodiments, logic blocks in memory chip 1300 may be connectedto a data bus and have an address identification. Process 2100 describesan address assignment method that disables faulty logic blocks andenables logic blocks that pass a test. The steps described in process2100 will be described as being performed by a production tester or aBIST executed at chip startup; however, other components of memory chip1300 and/or external devices may also perform one or more steps ofprocess 2100.

In step 2102, the tester may disable all logic and redundant blocks byassigning an illegal identification to each logic block at a chip level.

In step 2104, the tester may execute a testing protocol of a logicblock. For example, the tester may run testing methods described in step2002 for one or more of the logic blocks in memory chip 1300.

In step 2106, depending on the results of the test in step 2104, thetester may determine whether the logic block is defective. If the logicblock is not defective (step 2106: no), address manager may assign arunning ID to the tested logic block in step 2108. If the logic block isdefective (step 2106: yes), address manager 1302 may leave the illegalID for the defective logic block in step 2110.

In step 2112, address manager 1302 may select a redundant logic blockthat replicates the defective logic block. In some embodiments, theredundant logic block that replicates the defective logic block may havethe same components and connections to the defective logic blocks. Inother embodiments, however, the redundant logic block may have differentcomponents and/or connections to the defective logic blocks but be ableto perform an equivalent operation. For example, if the defective logicblock is designed to perform multiplication of vectors, the selectedredundant logic block would also be capable of performing multiplicationof vectors, even if it does not have the same architecture as thedefective unit.

In step 2114, address manager 1302 may test the redundant block. Forinstance, the tester may apply the testing techniques applied in step2104 to the identified redundant block.

In step 2116, based on the results of testing in step 2114, the testermay determine whether the redundant block is defective. In step 2118, ifthe redundant block is not defective (step 2116: no), the tester mayassign a running ID to the identified redundant block. In someembodiments, process 2100 may return to step 2104 after step 2118,creating an iteration loop to test all logic blocks in the memory chip.

If the tester determines the redundant block is defective (step 2116:yes), in step 2120, the tester may determine if additional redundantblocks are available. For example, the tester may query a memory bankwith information regarding available redundant logic blocks. Ifredundant logic blocks are available (step 2120: yes), the tester mayreturn to step 2112 and identify a new redundant logic block replicatingthe defective logic block. If redundant logic blocks are not available(step 2120: no), in step 2122, the tester may generate an error signal.The error signal may include information of the defective logic blockand the defective redundant block.

Coupled Memory Banks

The presently disclosed embodiments also include a distributedhigh-performance processor. The processor may include a memorycontroller that interfaces memory banks and processing units. Theprocessor may be configurable to expedite delivery of data to theprocessing units for calculations. For example, if a processing unitrequires two data instances to perform a task, the memory controller maybe configured so communication lines independently provide access to theinformation from two data instances. The disclosed memory architectureseeks to minimize hardware requirements that are associated with complexcache memory and complex register files schemes. Normally, processorchips include cache hierarchies that allow cores to work directly withregisters. However, the cache operations require significant die areaand consume additional power. The disclosed memory architecture avoidsthe use of a cache hierarchy by adding logic components in the memory.

The disclosed architecture also enables strategic (or even optimized)placement of data in memory banks Even if the memory banks have a singleport and high latency, the disclosed memory architecture may enable highperformance and avoid memory accessing bottlenecks by strategicallypositioning data in different blocks of memory banks With the goal ofproviding a continuous stream of data to the processing units, acompilation optimization step may determine how data should be stored inmemory banks for specific or generic tasks. Then, the memory controller,which interfaces processing units and memory banks, may be configured togrant access to specific processing units when they require data toperform operations.

The configuration of the memory chip may be performed by a processingunit (e.g., a configuration manager) or an external interface. Theconfiguration may be also written by a compiler or other SW tool. Inaddition, the configuration of the memory controller may be based on theavailable ports in the memory banks and the organization of data in thememory banks. Accordingly, the disclosed architecture may provideprocessing units with a constant flow of data or simultaneousinformation from different memory blocks. In this way, computation taskswithin the memory may be quickly processed by avoiding latencybottlenecks or cache memory requirements.

Moreover, data stored in the memory chip may be arranged based oncompilation optimization steps. The compilation may allow for buildingof processing routines in which the processor efficiently assigns tasksto processing units without memory latency associated delays. Thecompilation may be performed by a compiler and transmitted to a hostconnected to an external interface in the substrate. Normally, highlatency for certain access patterns and/or low numbers of ports wouldresult in data bottlenecks for processing units requiring the data. Thedisclosed compilation, however, may position data in memory banks in away that enables processing units to continuously receive data even withdisadvantageous memory types.

Furthermore, in some embodiments, a configuration manager may signalrequired processing units based on computations that are required by atask. Different processing units or logic blocks in the chip may havespecialized hardware or architectures for different tasks. Therefore,depending on the task that will be performed, a processing unit, or agroup of processing units, may be selected to perform the task. Thememory controller on the substrate may be configurable to route data, orgrant access, according to the selection of processing subunits toimprove data transfer rates. For example, based on the compilationoptimization and the memory architecture, processing units may begranted access to memory banks when they are required to perform a task.

Moreover, the chip architecture may include on-chip components thatfacilitate transfer of data by reducing the time required to access datain the memory banks Therefore, the present disclosure describes chiparchitecture(s), along with a compilation optimization step, for ahigh-performance processor capable of performing specific or generictasks using simple memory instances. The memory instances may have highlatency in random access and/or low number of ports, such as those usedin a DRAM device or other memory-oriented technologies, but thedisclosed architecture may overcome these shortcomings by enabling acontinuous (or nearly continuous) flow of data from memory banks toprocessing units.

In this application, simultaneous communication may refer tocommunication within a clock cycle. Alternatively, simultaneouscommunication may refer to sending information within a predetermineamount of time. For example, simultaneous communication may refer tocommunication within a few nanoseconds.

FIG. 22 provides block diagrams for exemplary processing devices,consistent with disclosed embodiments. FIG. 22A shows a first embodimentof a processing device 2200 in which a memory controller 2210 connects afirst memory block 2202 and a second memory block 2204 usingmultiplexers. Memory controller 2210 may also connect at least aconfiguration manager 2212, a logic block 2214, and multipleaccelerators 2216(a)-(n). FIG. 22B shows a second embodiment ofprocessing device 2200 in which memory controller 2210 connects memoryblocks 2202 and 2204 using a bus that connects memory controller 2210with at least a configuration manager 2212, a logic block 2214, andmultiple accelerators 2216(a)-(n). In addition, host 2230 may beexternal and connected to processing device 2200 through, for example,an external interface.

Memory blocks 2202 and 2204 may include a DRAM mats or group of mats,DRAM banks, MRAM\PRAM\RERAM\SRAM units, Flash mats, or other memorytechnologies. Memory blocks 2202 and 2204 may alternatively includenon-volatile memories, a flash memory device, a Resistive Random AccessMemory (ReRAM) device, or a Magnetoresistive Random Access Memory (MRAM)device.

Memory blocks 2202 and 2204 may additionally include a plurality ofmemory cells arranged in rows and columns between a plurality of wordlines (not shown) and a plurality of bit lines (not shown). The gates ofeach row of memory cells may be connected to a respective one of theplurality of word lines. Each column of memory cells may be connected toa respective one of the plurality of bit lines.

In other embodiments, a memory area (including memory blocks 2202 and2204) is built from simple memory instances. In this application, theterm “memory instance” may be used interchangeably with the term “memoryblock.” The memory instances (or blocks) may have poor characteristics.For example, the memories may be only one port memories and may havehigh random-access latency. Alternatively, or additionally, the memoriesmay be inaccessible during column and line changes and face data accessproblems related to, for example, capacity charging and/or circuitrysetups. Nonetheless, the architecture presented in FIG. 22 stillfacilitates parallel processing in the memory device by allowingdedicated connections between memory instances and processing units andarranging the data in a certain manner that takes the characteristics ofthe blocks into account.

In some device architectures, memory instances may include severalports, facilitating the parallel operations. Nonetheless, in suchembodiments, the chip may still achieve an improved performance whendata is compiled and organized based on the chip architecture. Forexample, a compiler may improve the efficiency of access in the memoryarea by providing instructions and organizing data placement, so it canbe readily access even using one-port memories.

Furthermore, memory blocks 2202 and 2204 may be multiple types formemory in a single chip. For example, memory blocks 2202 and 2204 may beeFlash and eDRAM. Also, memory blocks may include DRAM with instances ofROM.

Memory controller 2210 may include a logic circuit to handle the memoryaccess and return the results to the rest of the modules. For example,memory controller 2210 may include an address manager and selectiondevices, such as multiplexers, to route data between the memory blocksand processing units or grant access to the memory blocks.Alternatively, Memory controller 2210 may include double data rate (DDR)memory controllers used to drive DDR SDRAM, where data is transferred onboth rising and falling edges of the system's memory clock.

In addition, memory controller 2210 may constitute Dual Channel memorycontrollers. The incorporation of dual channel memory may facilitatecontrol of parallel access lines by memory controller 2210. The parallelaccess lines may be configured to have identical lengths to facilitatesynchronization of data when multiple lines are used in conjunction.Alternatively, or additionally, the parallel access lines may allowaccess of multiple memory ports of the memory banks.

In some embodiments processing device 2200 may include one or more muxesthat may be connected to processing units. The processing units mayinclude configuration manager 2212, logic block 2214, and accelerators2216, which may be connected directly to the mux. Also, memorycontroller 2210 may include at least one data input from a plurality ofmemory banks or blocks 2202 and 2204 and at least one data outputconnected to each one of the plurality of processing units. With thisconfiguration, memory controller 2210 may simultaneously receive datafrom memory banks or memory blocks 2202 and 2204 via the two datainputs, and simultaneously transmits data received via to the at leastone selected processing unit via the two data outputs. In someembodiments, however, the at least one data input and at least one dataoutput may be implemented in a single port allowing only read or writeoperations. In such embodiments, the single port may be implemented as adata bus including data, address, and command lines.

Memory controller 2210 may be connected to each one of the plurality ofmemory blocks 2202 and 2204, and may also connect to processing unitsvia, for example, a selection switch. Also processing units on thesubstrate, including configuration manager 2212, logic block 2214, andaccelerators 2216, may be independently connected to memory controller2210. In some embodiments, configuration manager 2212 may receive anindication of a task to be performed and, in response, configure memorycontroller 2210, accelerators 2216, and/or logic blocks 2214 accordingto a configuration stored in memory or supplied externally.Alternatively, memory controller 2210 may be configured by an externalinterface. The task may require at least one computation that may beused to select at least one selected processing unit from the pluralityof processing units. Alternatively, or additionally, the selection maybe based at least in part upon a capability of the selected processingunit for performing the at least one computation. In response, memorycontroller 2210 may grant access to the memory banks, or route databetween the at least one selected processing unit and at least twomemory banks, using dedicated buses and/or in a pipelined memory access.

In some embodiments, first memory block 2202 of at least two memoryblocks may be arranged on a first side of the plurality of processingunits; and second memory bank 2204 of the at least two memory banks maybe arranged on a second side of the plurality of processing unitsopposite to the first side. Further, a selected processing unit toperform the task, for instance accelerator 2216(n), may be configured toaccess second memory bank 2204 during a clock cycle in which acommunication line is opened to the first memory bank or first memoryblock 2202. Alternatively, the selected processing unit may beconfigured to transfer data to second memory block 2204 during a clockcycle in which a communication line is opened to first memory block2202.

In some embodiments, memory controller 2210 may be implemented as anindependent element, as shown in FIG. 22. In other embodiments, however,memory controller 2210 may be embedded in the memory area or may bedisposed along accelerators 2216(a)-(n).

A processing area in processing device 2200 may include configurationmanager 2212, logic block 2214, and accelerators 2216(a)-(n).Accelerators 2216 may include multiple processing circuits withpre-defined functions and may be defined by a specific application. Forexample, an accelerator may be a vector multiply accumulate (MAC) unitor a Direct Memory Access (DMA) unit handling memory moving betweenmodules. Accelerators 2216 may also be able to calculate their ownaddress and request the data from memory controller 2210 or write datato it. For example, configuration manager 2212 may signal at least oneof accelerators 2216 that he can access the memory bank. Thenaccelerators 2216 may configure memory controller 2210 to route data orgrant access to themselves. In addition, accelerators 2216 may includeat least one arithmetic logic unit, at least one vector handling logicunit, at least one string compare logic unit, at least one register, andat least one direct memory access.

Configuration manager 2212 may include digital processing circuits toconfigure accelerators 2216 and instructs execution of tasks. Forexample, configuration manager 2212 may be connected to memorycontroller 2210 and each one of the plurality of accelerators 2216.Configuration manager 2212 may have its own dedicated memory to hold theconfigurations of accelerators 2216. Configuration manager 2212 may usethe memory banks to fetch commands and configurations via memorycontroller 2210. Alternatively, configuration manager 2212 may beprogrammed through an external interface. In certain embodiments,configuration manager 2212 may be implemented with an on-chip reducedinstruction set computer (RISC) or an on-chip complex CPU with its owncache hierarchy. In some embodiments, configuration manager 2212 mayalso be omitted and the accelerators can be configured through anexternal interface.

Processing device 2200 may also include an external interface (notshown). The external interface allows access to the memory from an upperlevel, such a memory bank controller which receives the command fromexternal host 2230 or on-chip main processor or access to the memoryfrom external host 2230 or on-chip main processor. The externalinterface may allow programming of the configuration manager 2212 andthe accelerators 2216 by writing configurations or code to the memoryvia memory controller 2210 to be used later by configuration manager2212 or the units 2214 and 2216 themselves. The external interface,however, may also directly program processing units without being routedthrough memory controller 2210. In case configuration manager 2212 is amicrocontroller, configuration manager 2212 may allow loading of codefrom a main memory to the controller local memory via the externalinterface. Memory controller 2210 may be configured to interrupt thetask in response to receiving a request from the external interface.

The external interface may include multiple connectors associated withlogic circuits that provide a glue-less interface to a variety ofelements on the processing device. The external interface may include:Data I/O Inputs for data reads and output for data writes; Externaladdress outputs; External CEO chip select pins; Active-low chipselectors; Byte enable pins; a pin for wait states on the memory cycle;a Write enable pin; an Output enable-active pin; and read-write enablepin. Therefore, the external interface has the required inputs andoutputs to control processes and obtain information from the processingdevice. For example, the external interface may conform to JEDEC DDRstandards. Alternatively, or additionally, external interface mayconform to other standards such as SPI\OSPI or UART.

In some embodiments, the external interface may be disposed on the chipsubstrate and may be connected external host 2230. The external host maygain access to memory blocks 2202 and 2204, memory controller 2210, andprocessing units via the external interface. Alternatively, oradditionally, external host 2230 may read and write to the memory or maysignal configuration manager 2212, through read and write commands, toperform operations such as starting a process and/or stopping a process.In addition, external host 2230 may configure the accelerators 2216directly. In some embodiments, external host 2230 be able to performread/write operations directly on memory blocks 2202 and 2204.

In some embodiments, configuration manager 2212 and accelerators 2216may be configured to connect the device area with the memory area usingdirect buses depending on the target task. For example, a subset ofaccelerators 2216 may connect with memory instances 2204 when the subsetof accelerators has the capability to perform computations required toexecute the task. By doing such a separation, it is possible to assurethat dedicated accelerators get the bandwidth (BW) needed to memoryblocks 2202 and 2204. Moreover, this configuration with dedicated busesmay allow splitting a large memory to smaller instances or blocksbecause connecting memory instances to memory controller 2210 allowsquick access to data in different memories even with high row latencytime. To achieve the parallelization of connection, memory controller2210 may be connected to each of the memory instances with data,address, and/or control buses.

The above-discussed inclusion of memory controller 2210 may eliminatethe requirement of a cache hierarchy or complex register file in theprocessing device. Although the cache hierarchy can be added to giveadded capabilities, the architecture in processing device processingdevice 2200 may allow a designer to add enough memory blocks orinstances based on the processing operations and manage the instancesaccordingly without a cache hierarchy. For example, the architecture inprocessing device processing device 2200 may eliminate requirements of acache hierarchy by implementing a pipelined memory access. In thepipelined memory access, processing units may receive a sustaining flowof data in every cycle certain data lines may be opened (or activated)while other data lines receive or transmit data. The sustained flow ofdata using independent communication lines may allow an improvedexecution speed and minimum latency due to line changes.

Moreover, the disclosed architecture in FIG. 22 enables a pipelinedmemory access it may be possible to organize data in a low number ofmemory blocks and save power losses caused by line switching. Forexample, a In some embodiments, a compiler may communicate host 2230 theorganization of, or a method to organize, data in memory banks tofacilitate access to data during a given task. Then, configurationmanager 2212 may define which memory banks, and in some cases whichports of the memory banks, may be accessed by the accelerators. Thissynchronization between the location of data in memory banks and theaccess method to data, improves computing tasks by feeding data to theaccelerators with minimum latency. For example, in embodiments in whichconfiguration manager 2212 includes a RISC\CPU, the method may beimplemented in offline software (SW) and then the configuration manager2212 may be programmed to execute the method. The method may bedeveloped in any language executable by RISC/CPU computers and may beexecuted on any platform. The inputs of the method may includeconfiguration of the memories behind memory controller and the dataitself along with the pattern of memory accesses. In addition, themethod may be implemented in a language or machine language specific tothe embodiment and may also be just a series of configuration values inbinary or text.

As discussed above, in some embodiments, a compiler may provideinstructions to host 2230 for organizing data in memory blocks 2202 and2204 in preparation of a pipelined memory access. The pipelined memoryaccess may generally include steps of: receiving a plurality ofaddresses of a plurality of memory banks or memory blocks 2202 and 2204;accessing the plurality of memory banks according to the receivedaddresses using independent data lines; supplying data from a firstaddress through a first communication line to at least one of theplurality of processing units and opening a second communication line toa second address, the first address being in a first memory bank of theplurality of memory banks, the second address being in second memorybank 2204 of the plurality of memory banks; and supplying data from thesecond address through the second communication line to the at least oneof the plurality of processing units and opening a third communicationline to a third address in the first memory bank in the first linewithin a second clock cycle. In some embodiments, the pipelined memoryaccess may be executed with two memory blocks being connected to asingle port. In such embodiments, memory controller 2210 may hide thetwo memory blocks behind a single port but transmit data to theprocessing units with the pipelined memory access approach.

In some embodiments, a compiler can run on host 2230 before executing atask. In such embodiments, the compiler may be able to determine aconfiguration of data flow based on the architecture of the memorydevice since the configuration would be known to the compiler.

In other embodiments, if the configuration of memory blocks 2204 and2202 is unknown at offline time, the pipelined method can run on host2230 which may arrange data in memory blocks before startingcalculations. For example, host 2230 may directly write data in memoryblocks 2204 and 2202. In such embodiments, processing units, such asconfiguration manager 2212 and memory controller 2210 may not haveinformation regarding required hardware until run time. Then, it may benecessary to delay the selection of an accelerator 2216 until a taskstarts running. In these situations, the processing units or memorycontroller 2210 may randomly select an accelerator 2216 and create atest data access pattern, which may be modified as the task is executed.

Nonetheless, when the task is known in advance, a compiler may organizedata and instructions in memory banks for host 2230 to provide to aprocessing unit, such as configuration manager 2212, to set signalconnections that minimize access latency. For example, in some cases nwords may be needed at the same time by accelerators 2216. However, eachmemory instance supports retrieving only m words at a time, where “m”and “n” are integers and m<n. Thus, the compiler may place the neededdata across different memory instances or blocks facilitating dataaccess. Also, to avoid line miss latencies, a host may split data indifferent lines of different memory instances if processing device 2200includes multiple memory instances. The division of data may allowaccessing the next line of data in the next instance while still usingdata from the current instance.

For example, accelerator 2216(a) may be configured to multiply twovectors. Each one of the vectors may be stored in independent memoryblocks, such as memory blocks 2202 and 2204, and each vector may includemultiple words. Therefore, to complete a task requiring a multiplicationby accelerator 2216(a), it may be necessary to access the two memoryblocks and retrieve multiple words. However, in some embodiments, memoryblocks only allow access of one word per clock cycle. For instance,memory blocks may have a single port. In these cases, to expedite datatransmittal during an operation, a compiler may organize the wordscomposing vectors in different memory blocks allowing parallel and/orsimultaneous reading of the words. In these situations, a compiler maystore words in memory blocks that have a dedicated line. For instance,if each vector includes two words and memory controller has directaccess to four memory blocks, a compiler may arrange data in four memoryblocks, each one transmitting a word and expediting data delivery.Moreover, in embodiments when memory controller 2210 may have more thana single connection to each memory block, the compiler may instructconfiguration manager 2212 (or other processing unit) to access portsspecific ports. In this way, processing device 2200 may perform apipelined memory access, continuously providing data to processing unitsby simultaneously loading words in some lines and transmitting data inother lines. Thus, this pipelined memory access avoid may avoid latencyissues.

FIG. 23 is a block diagram of an exemplary processing device 2300,consistent with disclosed embodiments. The block diagram shows asimplified processing device 2300 displaying a single accelerator in theform of MAC Unit 2302, configuration manager 2304 (equivalent or similarto configuration manager 2212), memory controller 2306 (equivalent orsimilar to memory controller 2210), and a plurality of memory blocks2308(a)-(d).

In some embodiments, MAC unit 2302 may be a specific accelerator forprocessing a particular task. By way of example, the processing device2300 may be tasked with 2D-convolutions. Then, configuration manager2304 can signal an accelerator that has the appropriate hardware toperform calculations associated with the task. For instance, MAC unit2302 may have four internal incrementing counters (logical adders andregisters to manage the four loops needed by a convulsion calculation)and a multiply accumulate unit. Configuration manager 2304 may signalMAC unit 2302 to process incoming data and execute the task.Configuration manager 2304 may transmit an indication to MAC unit 2302to execute the task. In these situations, MAC unit 2302 may iterate overcalculated addresses, multiply the numbers, and accumulate them to aninternal register.

In some embodiments, configuration manager 2304 may configure theaccelerators while memory controller 2306 grants access to blocks 2308and MAC unit 2302 using dedicated buses. In other embodiments, however,memory controller 2306 can directly configure the accelerators based oninstructions received from configuration manger 2304 or an externalinterface. Alternatively, or additionally, configuration manager 2304can pre-load a few configurations and allow the accelerator toiteratively run on different addresses with different sizes. In suchembodiments, configuration manager 2304 may include a cache memory thatstores a command before it is transmitted to at least one of theplurality of processing units, such as accelerators 2216. However, inother embodiments configuration manager 2304 may not include a cache.

In some embodiments, configuration manager 2304 or memory controller2306 may receive addresses that need to be accessed for a task.Configuration manager 2304 or memory controller 2306 may check aregister to determine whether the address is already in a loaded line toone of memory blocks 2308. If so, memory controller 2306 may read theword from memory block 2308 and pass it to the MAC unit 2302. If theaddress is not in a loaded line, configuration manager 2304 may requestmemory controller 2306 may load the line and signal MAC unit 2302 todelay until it is retrieved.

In some embodiments, as shown in FIG. 23, memory controller 2306 mayinclude two inputs form two independent addresses. But if more than twoaddresses should be accessed simultaneously, and these addresses are ina single memory block (for example it is only in of memory blocks2308(a)), memory controller 2306 or configuration manager 2304 may raisean exception. Alternatively, configuration manager 2304 may returninvalid data signal when the two addresses can only be access through asingle line. In other embodiments, the unit may delay the processexecution until it is possible to retrieve all needed data. This maydiminish the overall performance. Nonetheless, a compiler may be able tofind a configuration and data placement that would prevent delays.

In some embodiments, a compiler may create a configuration orinstruction set for processing device 2300 that may configureconfiguration manager 2304 and memory controller 2306 and accelerator2302 to handle situations in which multiple addresses need to beaccessed from a single memory block but the memory block has one port.For instance, a compiler may re-arrange data in memory blocks 2308 suchthat processing units may access multiple lines in memory blocks 2308.

In addition, memory controller 2306 may also work simultaneously on morethan one input at the same time. For example, memory controller 2306 mayallow accessing one of memory blocks 2308 through one port and supplyingthe data while receiving a request from a different memory block inanother input. Therefore, this operation may result in and accelerator2216 tasked with the exemplary 2D-convolutions receiving data fromdedicated lines of communication with the pertinent memory blocks.

Additionally, or alternatively, memory controller 2306 or a logic blockmay hold refresh counters for every memory block 2308 and handle therefresh of all lines. Having such a counter allows memory controller2306 to slip in the refresh cycles between dead access times from thedevices.

Furthermore, memory controller 2306 may be configurable to perform thepipelined memory access, receiving addresses and opening lines in memoryblocks before supplying the data. The pipelined memory access mayprovide data to processing units without interruption or delayed clockcycles. For example, while memory controller 2306 or one of the logicblocks access data with the right line in FIG. 23, it may betransmitting data in the left line. These methods will be explained ingreater detail in connection to FIG. 26.

In response to the required data, processing device 2300 may usemultiplexors and/or other switching devices to choose which device getsserviced to perform a given task. For example, configuration manager2304 may configure multiplexers so at least two data lines reach the MACunit 2302. In this way, a task requiring data from multiple addresses,such as 2D-convolutions, may be performed faster because the vectors orwords requiring multiplication during convolution can reach theprocessing unit simultaneously, in a single clock. This datatransferring method may allow the processing units, such as accelerators2216, to quickly output a result.

In some embodiments, configuration manager 2304 may be configurable toexecute processes based on priority of tasks. For example, configurationmanager 2304 can be configured to let a running process finish withoutany interruptions. In that case, configuration manger 2304 may providean instruction or configurations of a task to accelerators 2216, letthem run uninterrupted, and switch multiplexers only when the task isfinished. However, in other embodiments, configuration manager 2304 mayinterrupt a task and reconfigure data routing when it receives apriority task, such a request from an external interface. Nevertheless,with enough memory blocks 2308, memory controller 2306 may beconfigurable to route data, or grant access, to processing units withdedicated lines that do not have to be changed until a task iscompleted. Moreover, in some embodiments, all devices may be connectedby buses to the entries of configuration manager 2304, and the devicesmay manage access between themselves and the buses (e.g., using the samelogic as a multiplexer). Therefore, memory controller 2306 may bedirectly connected to a number of memory instances or memory blocks.

Alternatively, memory controller 2306 may be connected directly tomemory sub-instances. In some embodiments, each memory instance or blockcan be built from sub-instances (for example, DRAM may be built frommats with independent data lines arranged in multiple sub-blocks).Further, the instances may include at least one of DRAM mats, DRAM,banks, flash mats, or SRAM mats or any other type of memory. Then,memory controller 2306 may include dedicated lines to addresssub-instances directly to minimize latency during a pipelined memoryaccess.

In some embodiments, memory controller 2306 may also hold the logicneeded for a specific memory instance (such as row\col decoders, refreshlogic, etc.) and memory blocks 2308 may handle its own logic. Therefore,memory blocks 2308 may get an address and generate commands forreturn\write data.

FIG. 24 depicts exemplary memory configuration diagrams, consistent withdisclosed embodiments. In some embodiments, a compiler generating codeor configuration for processing device 2200 may perform a method toconfigure loading from memory blocks 2202 and 2204 by pre-arranging datain each block. For example, a compiler may prearrange data so each wordrequired for a task is correlated to a line of memory instance or memoryblock(s). But for tasks that require more memory blocks than the oneavailable in processing device 2200, a compiler may implement methods offitting data in more than one memory location of each memory block. Thecompiler may also store data in sequence and evaluate the latency ofeach memory block to avoid line miss latency. In some embodiments, thehost may be part of a processing unit, such as configuration manger2212, but in other embodiments the compiler host may be connected toprocessing device 2200 via an external interface. In such embodiments,the host may run compiling functions, such as the ones described for thecompiler.

In some embodiments, configuration manager 2212 may be a CPU or a microcontroller (uC). In such embodiments, configuration manager 2212 mayhave to access the memory to fetch commands or instructions placed inthe memory. A specific compiler may generate the code and place it inthe memory in a manner that allows for consecutive commands to be storedin the same memory line and across a number of memory banks to allow forthe pipelined memory access also on the fetched command. In theseembodiments, configuration manager 2212 and memory controller 2210 maybe capable of avoiding row latency in linear execution by facilitatingthe pipelined memory access.

The previous case of linear execution of a program described a methodfor a compiler to recognize and place the instructions to allow forpipelined memory execution. However other software structures may bemore complex and would require the compiler to recognize them and actaccordingly. For example, in case a task requires loops and branches, acompiler may place all the loop code inside a single line so that thesingle line can be looped without line opening latency. Then, memorycontroller 2210 may not need to change lines during an execution.

In some embodiments, configuration manager 2212 may include internalcaching or small memory. The internal caching may store commands thatare executed by configuration manager 2212 to handle branches and loops.For example, commands in internal caching memory may includeinstructions to configure accelerators for accessing memory blocks.

FIG. 25 is an exemplary flowchart illustrating a possible memoryconfiguration process 2500, consistent with disclosed embodiments. Whereconvenient in describing memory configuration process 2500, referencemay be made to the identifiers of elements depicted in FIG. 22 anddescribed above. In some embodiments, process 2500 may be executed by acompiler that provides instructions to a host connected through anexternal interface. In other embodiments, process 2500 may be executedby components of processing device 2200, such as configuration manager2212.

In general, process 2500 may include determining a number of wordsrequired simultaneously to perform the task; determining a number ofwords that can be accessed simultaneously from each one of the pluralityof memory banks; and dividing the number of words requiredsimultaneously between multiple memory banks when the number of wordsrequired simultaneously is greater than the number of words that can beaccessed simultaneously. Moreover, dividing the number of words requiredsimultaneously may include executing a cyclic organization of words andsequentially assigning one word per memory bank.

More specifically, process 2500 may begin with step 2502, in which acompiler may receive a task specification. The specification includerequired computations and/or a priority level.

In step 2504, a compiler may identify an accelerator, or group ofaccelerators, that may perform the task. Alternatively, the compiler maygenerate instructions so the processing units, such as configurationmanager 2212, may identify an accelerator to perform the task. Forexample, using the required computation configuration manger 2212 mayidentify accelerators in the group of accelerators 2216 that may processthe task.

In step 2506, the compiler may determine a number of words that needs tobe simultaneously accessed to execute the task. For example, themultiplication of two vectors requires access to at least two vectors,and the compiler may therefore determine that vector words must besimultaneously accessed to perform the operation.

In step 2508, the compiler may determine a number of cycles necessary toexecute the task. For example, if the task requires a convolutionoperation of four by-products, the compiler may determine that at least4 cycles will be necessary to perform the task.

In step 2510, the compiler may place words that are needed to beaccessed simultaneously in different memory banks In that way, memorycontroller 2210 may be configured to open lines to different memoryinstances and access the required memory blocks within a clock cycle,without any required cached data.

In step 2512, the compiler place words that are accessed sequentially inthe same memory banks. For example, in the case that four cycles ofoperations are required, the compiler may generate instructions to writeneeded words in sequential cycles in a single memory block to avoidchanging lines between different memory blocks during execution.

In step 2514, compiler generate instructions for programing processingunits, such as configuration manager 2212. The instructions may specifyconditions to operate a switching device (such as a multiplexor) orconfigure a data bus. With such instructions, configuration manager 2212may configure memory controller 2210 to route data from, or grant accessto, memory blocks to processing units using dedicated lines ofcommunication according to a task.

FIG. 26 is an exemplary flowchart illustrating a memory read process2600, consistent with disclosed embodiments. Where convenient indescribing memory read process 2600, reference may be made to theidentifiers of elements depicted in FIG. 22 and described above. In someembodiments, as described below, process 2600 may be implemented bymemory controller 2210. In other embodiments, however, process 2600 maybe implemented by other elements in the processing device 2200, such asconfiguration manager 2212.

In step 2602, memory controller 2210, configuration manager 2212, orother processing units may receive an indication to route data from, orgrant access to, a memory bank. The request may specify an address and amemory block.

In some embodiments, the request may be received via a data busspecifying a read command in line 2218 and address in line 2220. Inother embodiments, the request may be received via demultiplexersconnected to memory controller 2210.

In step 2604, configuration manager 2212, a host, or other processingunits, may query an internal register. The internal register may includeinformation regarding opened lines to memory banks, opened addresses,opened memory blocks, and/or upcoming tasks. Based on the information inthe internal register, it may be determined whether there are linesopened to the memory bank and/or whether the memory block received therequest in step 2602. Alternatively, or additionally, memory controller2210 may directly query the internal register.

If the internal register indicates that the memory bank is not loaded inan opened line (step 2606: no), process 2600 may continue to step 2616and a line may be loaded to a memory bank associated with the receivedaddress. In addition, memory controller 2210 or a processing unit, suchas configuration manager 2212, may signal a delay to the elementrequesting information from the memory address in step 2616. Forexample, if accelerator 2216 is requesting the memory information thatis located an already occupied memory block, memory controller 2210 maysend a delay signal to the accelerator in step 2618. In step 2620,configuration manager 2212 or memory controller 2210 may update theinternal register to indicate a line has opened to a new memory bank ora new memory block.

If the internal register indicates that the memory bank is loaded in anopened line (step 2606: yes), process 2600 may continue to step 2608. Instep 2608, it may be determined whether the line loaded the memory bankis being used for a different address. If the line is being used for adifferent address (step 2608: yes), it would indicate that there are twoinstances in a single block and, therefore, they cannot be accessedsimultaneously. Thus, an error or exemption signal may be send to theelement requesting information from the memory address in step 2616.But, if the line is not being used for a different address (step 2608:no), a line may be opened for the address and retrieve data from thetarget memory bank and continue to step 2614 to transmit data to the tothe element requesting information from the memory address.

With process 2600, processing device 2200 has the ability to establishdirect connections between processing units and the memory blocks ormemory instances that contain the required information to perform atask. This organization of data would enable reading information fromorganized vectors in different memory instances, as well as allow theretrieval of information simultaneously from different memory blockswhen a device requests a plurality of these addresses.

FIG. 27 is an exemplary flowchart illustrating an execution process2700, consistent with disclosed embodiments. Where convenient indescribing execution process 2700, reference may be made to theidentifiers of elements depicted in FIG. 22 and described above.

In step 2702, a compiler or a local unit, such as configuration manager2212, may receive an indication of a task that needs to be performed.The task may include a single operation (e.g., multiplication) or a morecomplex operation (e.g., convolution between matrixes). The task mayalso indicate a required computation.

In step 2704, the compiler or configuration manager 2212 may determine anumber of words that is required simultaneously to perform the task. Forexample, configuration a compiler may determine two words are requiredsimultaneously to perform a multiplication between vectors. In anotherexample, a 2D convolution task, configuration manager 2212 may determinethat “n” times “m” words are required for a convolution betweenmatrices, where “n” and “m” are the matrices dimensions. Moreover, instep 2704, configuration manager 2212 may also determine a number ofcycles necessary to perform the task.

In step 2706, depending on the determinations in step 2704, a compilermay write words that need to be accessed simultaneously in a pluralityof memory banks disposed on the substrate. For instance, when a number anumber of words that can be accessed simultaneously from one of theplurality of memory banks is lower than the number of words that arerequired simultaneously, a compiler may organize data in multiple memorybanks to facilitate access to the different required words within aclock. Moreover, when configuration manager 2212 or the compilerdetermine a number of cycles is necessary to perform the task, thecompiler may write words that are needed in sequential cycles in asingle memory bank of the plurality of memory banks to prevent switchingof lines between memory banks.

In step 2708, memory controller 2210 may be configured to read or grantaccess to at least one first word from a first memory bank from theplurality of memory banks or blocks using a first memory line.

In step 2170, a processing unit, for example one of accelerators 2216,may process the task using the at least one first word.

In step 2712, memory controller 2210 may be configured to open a secondmemory line in a second memory bank. For example, based on the tasks andusing the pipelined memory access approach, memory controller 2210 maybe configured to open a second memory line in a second memory blockwhere information required for the tasks was written in step 2706. Insome embodiments, the second memory line may be opened when the task instep 2170 is about to be completed. For example, if a task requires 100clocks, the second memory line may be opened in the 90th clock.

In some embodiments, steps 2708-2712 may be executed within one lineaccess cycle.

In step 2714, memory controller 2210 may be configured to grant accessto data from at least one second word from the second memory bank usingthe second memory line opened in step 2710.

In step 2176, a processing unit, for example one of accelerators 2216,may process the task using the at least second word.

In step 2718, memory controller 2210 may be configured to open a secondmemory line in the first memory bank. For example, based on the tasksand using the pipelined memory access approach, memory controller 2210may be configured to open a second memory line to the first memoryblock. In some embodiments, the second memory line to the first blockmay be opened when the task in step 2176 is about to be completed.

In some embodiments, steps 2714-2718 may be executed within one lineaccess cycle.

In step 2720, memory controller 2210 may read or grant access to atleast one third word from the first memory bank from the plurality ofmemory banks or blocks using a second memory line in the first bank or afirst line in a third bank and continuing in different memory banks.

Some memory chips, such as dynamic random access memory (DRAM) chips,use refreshes to keep stored data (e.g., using capacitance) from beinglost due to voltage decay in capacitors or other electric components ofthe chips. For example, in DRAM each cell has to be refreshed from timeto time (based on the specific process and design) to restore the chargein the capacitors so that data is not lost or damaged. As the memorycapacities of a DRAM chip increase, the amount of time required torefresh the memory becomes significant. During the time periods when acertain line of memory is being refreshed, the bank containing the linebeing refreshed cannot be accessed. This can result in reductions inperformance. Additionally, the power associated with the refresh processmay also be significant. Prior efforts have attempted to reduce the rateat which refreshes are performed to reduce adverse effects associatedwith refreshing memory, but most of these efforts have focused on thephysical layers of the DRAM.

Refreshing is similar to reading and writing back a row of the memory.Using this principle and focusing on the access pattern to the memory,embodiments of the present disclosure include software and hardwaretechniques, as well as modifications to the memory chips, to use lesspower for refreshing and to reduce amounts of time during which memoryis refreshed. For example, an as an overview, some embodiments may usehardware and/or software to track line access timing and skip recentlyaccessed rows within a refresh cycle (e.g., based on a timingthreshold). In another example, some embodiments may rely on softwareexecuted by the memory chip's refresh controller to assign reads andwrites such that access to the memory is non-random. Accordingly, thesoftware may control the refresh more precisely to avoid wasted refreshcycles and/or lines. These techniques may be used alone or combined witha compiler that encodes commands for the refresh controller along withmachine code for a processor such that access to the memory is againnon-random. Using any combination of these techniques andconfigurations, which are described in detail below, the disclosedembodiments may reduce memory refresh power requirements and/or increasesystem performance by reducing an amount of time during which a memoryunit is refreshed.

FIG. 28 depicts an example memory chip 2800 with a refresh controller2803, consistent with the present disclosure. For example, memory chip2800 may include a plurality of memory banks (e.g., memory bank 2801 aand the like) on a substrate. In the example of FIG. 28, the substrateincludes four memory banks, each with four lines. A line may refer to awordline within one or more memory banks of memory chip 2800 or anyother collection of memory cells within memory chip 2800, such as aportion of or an entire row along a memory bank or a group of memorybanks.

In other embodiments, the substrate may include any number of memorybanks, and each memory bank may include any number of lines. Some memorybanks may include a same number of lines (as shown in FIG. 28) whileother memory banks may include different numbers of lines. As furtherdepicted in FIG. 28, memory chip 2800 may include a controller 2805 toreceive input to memory chip 2800 and transmit output from memory chip2800 (e.g., as described above in “Division of Code”).

In some embodiments, the plurality of memory banks may comprise dynamicrandom access memory (DRAM). However, the plurality of memory banks maycomprise any volatile memory that stores data requiring periodicrefreshes.

As will be discussed in more detail below, the presently disclosedembodiments may employ counters or resistor-capacitor circuits to timerefresh cycles. For example a counter or timer may be used to count timefrom the last full refresh cycle and then when the counter reaches itstarget value another counter may be used to iterate over all rows.Embodiments of the present disclosure may additionally track accesses tosegments of memory chip 2800 and reduce refresh power required. Forexample, although not depicted in FIG. 28, memory chip 2800 may furtherinclude a data storage configured to store access information indicativeof access operations for one or more segments of the plurality of memorybanks. For example, the one or more segments may comprise any portionsof lines, columns, or any other groupings of memory cells within memorychip 2800. In one particular example, the one or more segments mayinclude at least one row of memory structures within the plurality ofmemory banks Refresh controller 2803 may be configured to perform arefresh operation of the one or more segments based, at least in part,on the stored access information.

For example, the data storage may comprise one or more registers, staticrandom access memory (SRAM) cells, or the like associated with segmentsof memory chip 2800 (e.g., lines, columns, or any other groupings ofmemory cells within memory chip 2800). Further, the data storage may beconfigured to store bits indicative of whether the associated segmentwas accessed in one or more previous cycles. A “bit” may comprise anydata structure storing at least one bit, such as a register, an SRAMcell, a nonvolatile memory, or the like. Moreover, a bit may be set bysetting a corresponding switch (or switching element, such as atransistor) of the data structure to ON (which may be equivalent to “1”or “true”). Additionally or alternatively, a bit may be set by modifyingany other property within the data structure (such as charging afloating gate of a flash memory, modifying a state of one or moreflip-flops in an SRAM, or the like) in order to write a “1” to the datastructure (or any other value indicating the setting of a bit). If a bitis determined to be set as part of the memory controller's refreshoperation, refresh controller 2803 may skip a refresh cycle for theassociated segment and clear the register(s) associated with thatportion.

In another example, the data storage may comprise one or morenonvolatile memories (e.g., a flash memory or the like) associated withsegments of memory chip 2800 (e.g., lines, columns, or any othergroupings of memory cells within memory chip 2800). The nonvolatilememory may be configured to store bits indicative of whether theassociated segment was accessed in one or more previous cycles.

Some embodiments may additionally or alternatively add a timestampregister on each row or group of rows (or other segment of memory chip2800) holding the last tick within the current refresh cycle which theline was accessed. This means that with each row access, the refreshcontroller may update the row timestamp register. Thus, when a next timeto refresh occurs (e.g., at the end of a refresh cycle), the refreshcontroller may compare the stored timestamp, and if the associatedsegment was previously accessed within a certain period of time (e.g.,within a certain threshold as applied to the stored timestamp), therefresh controller may skip to the next segment. This saves the systemfrom expending refresh power on segments that have been recentlyaccessed. Moreover, the refresh controller may continue to track accessto make sure each segment is accessed or refreshed at the next cycle.

Accordingly, in yet another example, the data storage may comprise oneor more registers or nonvolatile memories associated with segments ofmemory chip 2800 (e.g., lines, columns, or any other groupings of memorycells within memory chip 2800). Rather than using bits to indicatewhether an associated segment has been accessed, the registers ornonvolatile memories may be configured to store timestamps or otherinformation indicative of a most recent access of the associatedsegments. In such an example, refresh controller 2803 may determinewhether to refresh or access the associated segments based on whether anamount of time between timestamps stored in the associated registers ormemories and a current time (e.g., from a timer, as explained below inFIGS. 29A and 29B) exceed a predetermined threshold (e.g., 8 ms, 16 ms,32 ms, 64 ms, or the like).

Accordingly, the predetermined threshold may comprise an amount of timefor a refresh cycle to ensure that the associated segments are refreshed(if not accessed) at least once per refresh cycle. Alternatively, thepredetermined threshold may comprise an amount of time shorter than thatrequired for a refresh cycle (e.g., to ensure that any required refreshor access signals may reach the associated segments before the refreshcycle is complete). For example, the predetermined time may comprise 7ms for a memory chip with an 8 ms refresh period such that, if a segmenthas not been accessed in 7 ms, the refresh controller will send arefresh or access signal that reaches the segment by the end of the 8 msrefresh period. In some embodiments, the predetermined threshold maydepend on the size of an associated segment. For example, thepredetermined threshold may be smaller for smaller segments of memorychip 2800.

Although described above with respect to a memory chip, the refreshcontrollers of the present disclosure may also be used in distributedprocessor architectures, like those described in the sections above andthroughout the present disclosure. One example of such an architectureis depicted in FIG. 7A. In such embodiments, the same substrate asmemory chip 2800 may include, disposed thereon, a plurality ofprocessing groups, e.g., as depicted in FIG. 7A. As explained above withrespect to FIG. 3A, a “processing group” may refer to two or moreprocessor subunits and their corresponding memory banks on thesubstrate. The group may represent a spatial distribution on thesubstrate and/or a logical grouping for the purposes of compiling codefor execution on memory chip 2800. Accordingly, the substrate mayinclude a memory array that includes a plurality of banks, such as banks2801 a and other banks shown in FIG. 28. Furthermore, the substrate mayinclude a processing array that may include a plurality of processorsubunits (such as subunits 730 a, 730 b, 730 c, 730 d, 730 e, 730 f, 730g, and 730 h shown in FIG. 7A).

As further explained above with respect to FIG. 7A, each processinggroup may include a processor subunit and one or more correspondingmemory banks dedicated to the processor subunit. Moreover, to allow eachprocessor subunit to communicate with its corresponding, dedicatedmemory bank(s), the substrate may include a first plurality of busesconnecting one of the processor subunits to its corresponding, dedicatedmemory bank(s).

In such embodiments, as shown in FIG. 7A, the substrate may include asecond plurality of buses to connect each processor subunit to at leastone other processor subunit (e.g., an adjacent subunit in in the samerow, an adjacent processor subunit in the same column, or any otherprocessor subunit on the substrate). The first and/or second pluralityof buses may be free of timing hardware logic components such that datatransfers between processor subunits and across corresponding ones ofthe plurality of buses are uncontrolled by timing hardware logiccomponents, as explained above in the “Synchronization Using Software”section.

In embodiments where same substrate as memory chip 2800 may include,disposed thereon, a plurality of processing groups (e.g., as depicted inFIG. 7A), the processor subunits may further include an addressgenerator (e.g., address generator 450 as depicted in FIG. 4). Moreover,each processing group may include a processor subunit and one or morecorresponding memory banks dedicated to the processor subunit.Accordingly, each one of the address generators may be associated with acorresponding, dedicated one of the plurality of memory banks. Inaddition, the substrate may include a plurality of buses, eachconnecting one of the plurality of address generators to itscorresponding, dedicated memory bank.

FIG. 29A depicts example refresh controller 2900 consistent with thepresent disclosure. Refresh controller 2900 may be incorporated in amemory chip of the present disclosure, such as memory chip 2800 of FIG.28. As depicted in FIG. 29A, refresh controller 2900 may include a timer2901, which may comprise a on-chip oscillator or any other timingcircuit for refresh controller 2900. In the configuration depicted inFIG. 29A, timer 2901 may trigger a refresh cycle periodically (e.g.,every 8 ms, 16 ms, 32 ms, 64 ms, or the like). The refresh cycle may usea row counter 2903 to cycle through all rows of a corresponding memorychip and generate a refresh signal for each row using adder 2901combined with an active bit 2905. As shown in FIG. 29A, bit 2905 may befixed at 1 (“true”) to ensure that each row is refreshed during a cycle.

In embodiments of the present disclosure, refresh controller 2900 may beinclude a data storage. As described above, the data storage maycomprise one or more registers or nonvolatile memories associated withsegments of memory chip 2800 (e.g., lines, columns, or any othergroupings of memory cells within memory chip 2800). The registers ornonvolatile memories may be configured to store timestamps or otherinformation indicative of a most recent access of the associatedsegments.

Refresh controller 2900 may use the stored information to skip refreshesfor segments of memory chip 2900. For example, refresh controller 2900may skip a segment in a current refresh cycle if the informationindicates it was refreshed during one or more previous refresh cycles.In another example, refresh controller 2900 may skip a segment in acurrent refresh cycle if a difference between the stored timestamp forthe segment a current time is below a threshold. Refresh controller 2900may further continue to track accesses and refreshes of the segments ofmemory chip 2800 through multiple refresh cycles. For example, refreshcontroller 2900 may update stored timestamps using timer 2901. In suchembodiments, refresh controller 2900 may be configured to use an outputof the timer in clearing the access information stored in the datastorage after a threshold time interval. For example, in embodimentswhere the data storage stores timestamps of a most recent access orrefresh for an associated segment, refresh controller 2900 may store anew timestamp in the data storage whenever an access command or refreshsignal is sent to the segment. If the data storage stores bits ratherthan timestamps, timer 2901 may be configured to clear bits that are setfor longer than a threshold period of time. For example, in embodimentswhere the data storage stores bits indicating that associated segmentswas accessed in one or more previous cycles, refresh controller 2900 mayclear bits (e.g., setting them to 0) in the data storage whenever timer2901 triggers a new refresh cycle that is a threshold number of cycles(e.g., one, two, or the like) later since the associated bits were set(e.g., set to 1).

Refresh controller 2900 may track access of the segments of memory chip2800 in cooperation with other hardware of memory chip 2800. Forexample, memory chips use sense amplifiers to perform read operations(e.g., as shown above in FIGS. 9 and 10). The sense amplifiers maycomprise a plurality of transistors configured to sense low-powersignals from a segment of the memory chip 2800 storing a data in one ormore memory cells and amplify the small voltage swing to higher voltagelevels such that the data can be interpreted by logic, such as externalCPUs or GPUs or integrated processor subunits as explained above.Although not depicted in FIG. 29A, refresh controller 2900 may furthercommunicate with a sense amplifier configured to access the one or moresegments and change the state of the at least one bit register. Forexample, when the sense amplifier accesses the one or more segments, itmay set (e.g., set to 1) bits associated with the segments indicatingthat the associated segments were accessed in a previous cycle. Inembodiments where the data storage stores timestamps of a most recentaccess or refresh for an associated segment, when the sense amplifieraccesses the one or more segments, it may trigger a write of a timestampfrom timer 2901 to the registers, memories, or other elements comprisingthe data storage.

In any of the embodiments described above, refresh controller 2900 maybe integrated with a memory controller for the plurality of memorybanks. For example, similar to the embodiments depicted in FIG. 3A,refresh controller 2900 may be incorporated into a logic and controlsubunit associated with a memory bank or other segment of memory chip2800.

FIG. 29B depicts another example refresh controller 2900′ consistentwith the present disclosure. Refresh controller 2900′ may beincorporated in a memory chip of the present disclosure, such as memorychip 2800 of FIG. 28. Similar to refresh controller 2900, refreshcontroller 2900′ includes timer 2901, row counter 2903, active bit 2905,and adder 2907. Additionally, refresh controller 2900′ may include datastorage 2909. As shown in FIG. 29B, data storage 2909 may comprise oneor more registers or nonvolatile memories associated with segments ofmemory chip 2800 (e.g., lines, columns, or any other groupings of memorycells within memory chip 2800), and states within the data storage maybe configured to be changed (e.g., by a sense amplifier and/or otherelements of refresh controller 2900′, as described above) in response tothe one or more segments being accessed. Accordingly, the refreshcontroller 2900′ may be configured to skip a refresh of the one or moresegments based on the states within the data storage. For example, if astate associated with a segment is activated (e.g., set to 1 by beingswitched on, having a property altered in order to store a “1,” or thelike), refresh controller 2900′ may skip a refresh cycle for theassociated segment and clear the state associated with that portion. Thestate may be stored with at least a one-bit register or any other memorystructure configured to store at least one bit of data.

In order to ensure segments of the memory chip are refreshed or accessedduring each refresh cycle, refresh controller 2900′ may reset orotherwise clear the states in order to trigger a refresh signal duringthe next refresh cycle. In some embodiments, after a segment is skipped,refresh controller 2900′ may clear the associated state in order toensure that the segment is refreshed on the next refresh cycle. In otherembodiments, refresh controller 2900′ may be configured to reset thestates within the data storage after a threshold time interval. Forexample, refresh controller 2900′ may clear states (e.g., setting themto 0) in the data storage whenever timer 2901 exceeds a threshold timesince the associated states were set (e.g., set to 1 by being switchedon, having a property altered in order to store a “1,” or the like). Insome embodiments, refresh controller 2900′ may use a threshold number ofrefresh cycles (e.g., one, two, or the like) or use a threshold numberof clock cycles (e.g., two, four, or the like) rather than a thresholdtime.

In other embodiments, the state may comprise a timestamp of a mostrecent refresh or access of an associated segment such that, if anamount of time between the timestamp and a current time (e.g., fromtimer 2901 of FIGS. 29A and 29B) exceeds a predetermined threshold(e.g., 8 ms, 16 ms, 32 ms, 64 ms, or the like), refresh controller 2900′may send an access command or a refresh signal to the associated segmentand update the timestamp associated with that portion (e.g., using timer2901). Additionally or alternatively, refresh controller 2900′ may beconfigured to skip a refresh operation relative to the one or moresegments of the plurality of memory banks if the refresh time indicatorindicates a last refresh time within a predetermined time threshold. Insuch embodiments, refresh controller 2900′, after skipping a refreshoperation relative to the one or more segments, may be configured toalter the stored refresh time indicator associated with the one or moresegments such that during a next operation cycle, the one or moresegments will be refreshed. For example, as described above, refreshcontroller 2900′ may use timer 2901 to update the stored refresh timeindicator.

Accordingly, the data storage may include a timestamp registerconfigured to store a refresh time indicator indicative of a time atwhich the one or more segments of the plurality of memory banks werelast refreshed. Moreover, refresh controller 2900′ may use an output ofthe timer in clearing the access information stored in the data storageafter a threshold time interval.

In any of the embodiments described above, access to the one or moresegments may include a write operation associated with the one or moresegments. Additionally or alternatively, access to the one or moresegments may include a read operation associated with the one or moresegments.

Moreover, as depicted in FIG. 29B, refresh controller 2900′ may comprisea row counter 2903 and an adder 2907 configured to assist in updatingthe data storage 2909 based, at least in part, on the states within thedata storage. Data storage 2909 may comprise a bit table associated withthe plurality of memory banks. For example, the bit table may comprisean array of switches (or switching elements such as transistors) orregisters (e.g., SRAM or the like) configured to hold bits forassociated segments. Additionally or alternatively, data storage 2909may store timestamps associated with the plurality of memory banks.

Moreover, refresh controller 2900′ may include a refresh gate 2911configured to control whether a refresh to the one or more segmentsoccurs based a corresponding value stored in the bit table. For example,refresh gate 2911 may comprise a logic gate (such as an “and” gate)configured to nullify a refresh signal from row counter 2903 if acorresponding state of data storage 2909 indicates that the associatedsegment was refreshed or accessed during one or more previous clockcycles. In other embodiments, refresh gate 2911 may comprise amicroprocessor or other circuit configured to nullify a refresh signalfrom row counter 2903 if a corresponding timestamp from data storage2909 indicates that the associated segment was refreshed or accessedwithin a predetermined threshold time value.

FIG. 30 is an example flowchart of a process 3000 for partial refreshesin a memory chip (e.g., memory chip 2800 of FIG. 28) Process 3000 may beexecuted by a refresh controller consistent with the present disclosure,such as refresh controller 2900 of FIG. 29A or refresh controller 2900′of FIG. 29B.

At step 3010, the refresh controller may access information indicativeof access operations for one or more segments of a plurality of memorybanks. For example, as explained above with respect to FIGS. 29A and29B, the refresh controller may include a data storage associated withsegments of memory chip 2800 (e.g., lines, columns, or any othergroupings of memory cells within memory chip 2800) and configured tostore timestamps or other information indicative of a most recent accessof the associated segments.

At step 3020, the refresh controller may generate refresh and/or accesscommands based, at least in part, on the accessed information. Forexample, as explained above with respect to FIGS. 29A and 29B, therefresh controller may skip a refresh operation relative to the one ormore segments of the plurality of memory banks if the accessedinformation indicates a last refresh or access time within apredetermined time threshold and/or if the accessed informationindicates a last refresh or access occurred during one or more previousclock cycles. Additionally or alternatively, the refresh controller maygenerate comments to refresh or access the associated segments based onwhether the accessed information indicates a last refresh or access timethat exceeds a predetermined threshold and/or if the accessedinformation indicates a last refresh or access did not occur during oneor more previous clock cycles.

At step 3030, the refresh controller may alter the stored refresh timeindicator associated with the one or more segments such that during anext operation cycle, the one or more segments will be refreshed. Forexample, after skipping a refresh operation relative to the one or moresegments, the refresh controller may alter the information indicative ofaccess operations for the one or more segments such that, during a nextclock cycle, the one or more segments will be refreshed. Accordingly,the refresh controller may clear (e.g., set to 0) states for thesegments after skipping a refresh cycle. Additionally or alternatively,the refresh controller may set (e.g., set to 1) states for the segmentsthat are refreshed and/or accessed during the current cycle. Inembodiments where the information indicative of access operations forthe one or more segments includes timestamps, the refresh controller mayupdate any stored timestamps associated with segments that are refreshedand/or accessed during the current cycle.

Method 3000 may further include additional steps. For example, inaddition to or as an alternative to step 3030, a sense amplifier mayaccess the one or more segments and may change the informationassociated with the one or more segments. Additionally or alternatively,the sense amplifier may signal to the refresh controller when the accesshas occurred such that the refresh controller may update the informationassociated with the one or more segments. As explained above, a senseamplifier may comprise a plurality of transistors configured to senselow-power signals from a segment of the memory chip storing a data inone or more memory cells and amplify the small voltage swing to highervoltage levels such the data can be interpreted by logic, such asexternal CPUs or GPUs or integrated processor subunits as explainedabove. In such an example, whenever the sense amplifier accesses the oneor more segments, it may set (e.g., set to 1) bits associated with thesegments indicating that the associated segments were accessed in aprevious cycle. In embodiments where the information indicative ofaccess operations for the one or more segments includes timestamps,whenever the sense amplifier accesses the one or more segments, it maytrigger a write of a timestamp from a timer of the refresh controller tothe data storage to update any stored timestamps associated with thesegments.

FIG. 31 is an example flowchart of a process 3100 for determiningrefreshes for a memory chip (e.g., memory chip 2800 of FIG. 28). Process3100 may be implemented within a compiler consistent with the presentdisclosure. As explained above, a “compiler” refers to any computerprogram that converts a higher-level language (e.g., a procedurallanguage, such as C, FORTRAN, BASIC, or the like; an object-orientedlanguage, such as Java, C++, Pascal, Python, or the like; etc.) to alower-level language (e.g., assembly code, object code, machine code, orthe like). The compiler may allow a human to program a series ofinstructions in a human-readable language, which is then converted to amachine-executable language. The compiler may comprise softwareinstructions executed by one or more processors.

At step 3110, the one or more processors may receive higher-levelcomputer code. For example, the higher-level computer code may beencoded in one or more files on a memory (e.g., a non-volatile memorysuch as a hard disk drive or the like, a volatile memory such as DRAM,or the like) or received over a network (e.g., the Internet or thelike). Additionally or alternatively, the higher-level computer code maybe received from a user (e.g., using an input device such as akeyboard).

At step 3120, the one or more processors may identify a plurality ofmemory segments distributed over a plurality of memory banks associatedwith a memory chip to be accessed by the higher-level computer code. Forexample, the one or more processors may access a data structure definingthe plurality of memory banks and a corresponding structure of thememory chip. The one or more processor may access the data structurefrom a memory (e.g., a non-volatile memory such as a hard disk drive orthe like, a volatile memory such as DRAM, or the like) or receive thedata structure over a network (e.g., the Internet or the like). In suchembodiments, the data structure may be included in one or more librariesaccessible by the compiler to permit the compiler to generateinstructions for the particular memory chip to be accessed.

At step 3130, the one or processors may assess the higher-level computercode to identify a plurality of memory read commands to occur over aplurality of memory access cycles. For example, the one or moreprocessor may identify each operation within the higher-level computercode requiring one or more read commands from memory and/or one or morewrite commands to memory. Such instructions may include variableinitialization, variable re-assignment, logic operations on variables,input-output operations, or the like.

At step 3140, the one or more processors may cause a distribution ofdata, associated with the plurality of memory access commands, acrosseach of the plurality of memory segments such that each of the pluralityof memory segments is accessed during each of the plurality of memoryaccess cycles. For example, the one or more processors may identify thememory segments from the data structure defining the structure of thememory chip and then assign variables from the higher-level code tovarious ones of the memory segments such that each memory segment isaccessed (e.g., via a write or a read) at least once during each refreshcycle (which may comprise a particular number of clock cycles). In suchan example, the one or more processors may access information indicativeof how many clock cycles each line of higher-level code requires inorder to assign variables from the lines of higher-level code such thateach memory segment is accessed (e.g., via a write or a read) at leastonce during the particular number of clock cycles.

In another example, the one or more processors may first generatemachine code or other lower-level code from the higher-level code. Theone or more processors may then assign variables from the lower-levelcode to various ones of the memory segments such that each memorysegment is accessed (e.g., via a write or a read) at least once duringeach refresh cycle (which may comprise a particular number of clockcycles). In such an example, each line of lower-level code may require asingle clock cycle.

In any of the examples given above, the one or more processor mayfurther assign logic operations or other commands that use temporaryoutput to various ones of the memory segments. Such temporary outputsmay still result in read and/or write commands such that the assignedmemory segment is still being accessed during that refresh cycle eventhough a named variable has not been assigned to that memory segment.

Method 3100 may further include additional steps. For example, the oneor more processors may, in embodiments where the variables are assignedprior to compiling, generate machine code or other lower-level code fromthe higher-level code. Moreover, the one or more processors may transmitthe compiled code for execution by the memory chip and correspondinglogic circuits. The logic circuits may comprise conventional circuitssuch as GPUs or CPUs or may comprise processing groups on the samesubstrate as the memory chip, e.g., as depicted in FIG. 7A. Accordingly,as described above, the substrate may include a memory array thatincludes a plurality of banks, such as banks 2801 a and other banksshown in FIG. 28. Furthermore, the substrate may include a processingarray that may include a plurality of processor subunits (such assubunits 730 a, 730 b, 730 c, 730 d, 730 e, 730 f, 730 g, and 730 hshown in FIG. 7A).

FIG. 32 is another example flowchart of a process 3200 for determiningrefreshes for a memory chip (e.g., memory chip 2800 of FIG. 28). Process3200 may be implemented within a compiler consistent with the presentdisclosure. Process 3200 may be executed by one or more processorsexecuting software instructions comprising the compiler. Process 3200may be implemented separately from or in combination with process 3100of FIG. 31.

At step 3210, similar to step 3110, the one or more processors mayreceive higher-level computer code. At step 3220, similar to step 3210,the one or more processors may identify a plurality of memory segmentsdistributed over a plurality of memory banks associated with a memorychip to be accessed by the higher-level computer code.

At step 3230, the one or more processors may assess the higher-levelcomputer code to identify a plurality of memory read commands eachimplicating one or more of the plurality of memory segments. Forexample, the one or more processor may identify each operation withinthe higher-level computer code requiring one or more read commands frommemory and/or one or more write commands to memory. Such instructionsmay include variable initialization, variable re-assignment, logicoperations on variables, input-output operations, or the like.

In some embodiments, the one or more processors may simulate anexecution of the higher-level code using logic circuits and theplurality of memory segments. For example, the simulation may comprise aline-by-line step-through of the higher-level code similar to that of adebugger or other instruction set simulator (ISS). The simulation mayfurther maintain internal variables which represent the addresses of theplurality of memory segments, similar to how a debugger may maintaininternal variables which represent registers of a processor.

At step 3240, the one or more processors may, based on analysis of thememory access commands and for each memory segment among the pluralityof memory segments, track an amount of time that would accrue from alast access to the memory segment. For example, using the simulationdescribed above, the one or processors may determine lengths of timebetween each access (e.g., a read or a write) to one or more addresseswithin each of the plurality of memory segments. The lengths of time maybe measured in absolute time, clock cycles, or refresh cycles (e.g.,determined by a known refresh rate of the memory chip).

At step 3250, in response to a determination that an amount of timesince a last access for any particular memory segment would exceed apredetermined threshold, the one or more processors may introduce intothe higher-level computer code at least one of a memory refresh commandor a memory access command configured to cause an access to theparticular memory segment. For example, the one or more processors mayinclude a refresh command for execution by a refresh controller (e.g.,refresh controller 2900 of FIG. 29A or refresh controller 2900′ of FIG.29B). In embodiments where the logic circuits are not embedded on thesame substrate as the memory chip, the one or more processors maygenerate the refresh commands for sending to the memory chip separatefrom the lower-level code for sending to the logic circuits.

Additionally or alternatively, the one or more processors may include anaccess command for execution by a memory controller (which may beseparate from the refresh controller or incorporated into the same). Theaccess command may comprise a dummy command configured to trigger a readoperation on the memory segment but without having the logic circuitsperform any further operation on the read or written variable from thememory segment.

In some embodiments, the compiler may include a combination of stepsfrom process 3100 and from process 3200. For example, the compiler mayassign variables according to step 3140 and then run the simulationdescribed above to add in any additional memory refresh commands ormemory access commands according to step 3250. This combination mayallow for the compiler to distribute the variables across as many memorysegments as possible and to generate refresh or access commands for anymemory segments that cannot be accessed within the predeterminedthreshold amount of time. In another combinatory example, the compilermay simulate the code according to step 3230 and assign variablesaccording to step 3140 based on any memory segments that the simulationindicates will not be accessed within the predetermined threshold amountof time. In some embodiments, this combination may further include step3250 to allow for the compiler to generate refresh or access commandsfor any memory segments that cannot be accessed within the predeterminedthreshold amount of time, even after assignments according to step 3140are complete.

Refresh controllers of the present disclosure may allow softwareexecuted by logic circuits (whether conventional logic circuits such asCPUs and GPUs or processing groups on the same substrate as the memorychip, e.g., as depicted in FIG. 7A) to disable an automatic refreshexecuted by the refresh controller and control the refresh via theexecuted software instead. Accordingly, some embodiments of the presentdisclosure may provide software with a known access pattern to a memorychip (e.g., if the compiler has access to a data structure defining aplurality of memory banks and a corresponding structure of the memorychip). In such embodiments, a post-compiling optimizer may disableautomatic refresh and manually set refresh controls only for segments ofthe memory chip not accessed within threshold amounts of time. Thus,similar to step 3250 described above but after compilation, thepost-compiling optimizer may generate refresh commands to ensure eachmemory segment is accessed or refreshed with the predetermined thresholdamount of time.

Another example of reducing refresh cycles may include using predefinedpatterns of access to the memory chip. For example, if software executedby the logic circuits can control its access pattern for the memorychip, some embodiments may create access patterns for refresh beyondconventional linear line refreshes. For example, if a controllerdetermines that software executed by the logic circuits accessesregularly every second row of memory, then a refresh controller of thepresent disclosure may use an access pattern that does not refresh everysecond line in order to speed up the memory chip and reduce power usage.

An example of such a refresh controller is shown in FIG. 33. FIG. 33depicts an example refresh controller 3300 configured by stored patternsconsistent with the present disclosure. Refresh controller 3300 may beincorporated in a memory chip of the present disclosure, e.g., having aplurality of memory banks and a plurality of memory segments included ineach of the plurality of memory banks, such as memory chip 2800 of FIG.28.

Refresh controller 3300 includes a timer 3301 (similar to timer 2901 ofFIGS. 29A and 29B), a row counter 3303 (similar to row counter 2903 ofFIGS. 29A and 29B), and an adder 3305 (similar to adder 2907 of FIGS.29A and 29B). Moreover, refresh controller 3300 includes a data storage3307. Unlike data store 2909 of FIG. 29B, data storage 3307 may store atleast one memory refresh pattern to be implemented in refreshing theplurality of memory segments included in each of the plurality of memorybanks. For example, as depicted in FIG. 33, data storage 3307 mayinclude Li (e.g., L1, L2, L3, and L4 in the example of FIG. 33) and Hi(e.g., H1, H2, H3, and H4 in the example of FIG. 33) that definesegments in the memory banks by row and/or column. Moreover, eachsegment may be associated with an Inci variable (e.g., Inc1, Inc2, Inc3,and Inc4 in the example of FIG. 33) which defines how the rowsassociated with the segment are incremented (e.g., whether each row isaccessed or refresh, whether every other row is accessed or refreshed,or the like). Thus, as shown in FIG. 33, the refresh pattern maycomprise a table including a plurality of memory segment identifiersassigned by the software to identify ranges of the plurality of memorysegments in a particular memory bank that are to be refreshed during arefresh cycle and ranges of the plurality of memory segments in theparticular memory bank that are not to be refreshed during the refreshcycle.

Thus, data storage 3308 may define a refresh pattern which the softwareexecuted by logic circuits (whether conventional logic circuits such asCPUs and GPUs or processing groups on the same substrate as the memorychip, e.g., as depicted in FIG. 7A) may select for use. The memoryrefresh pattern may be configurable using software to identify which ofthe plurality of memory segments in a particular memory bank are to berefreshed during a refresh cycle and which of the plurality of memorysegments in the particular memory bank are not to be refreshed duringthe refresh cycle. Thus, refresh controller 3300 may refresh some or allrows within the defined segments that are not accessed during a currentcycle according to Inci. Refresh controller 3300 may skip other rows ofthe defined segments that are set for access during the current cycle.

In embodiments where data storage 3308 of refresh controller 3300includes a plurality of memory refresh patterns, each may represent adifferent refresh pattern for refreshing the plurality of memorysegments included in each of the plurality of memory banks. The memoryrefresh patterns may be selectable for use on the plurality of memorysegments. Accordingly, refresh controller 3300 may be configured toallow selection of which of the plurality of memory refresh patterns toimplement during a particular refresh cycle. For example, the softwareexecuted by logic circuits (whether conventional logic circuits such asCPUs and GPUs or processing groups on the same substrate as the memorychip, e.g., as depicted in FIG. 7A) may select different memory refreshpatterns for use during one or more different refresh cycles.Alternatively, the software executed by logic circuits may select onememory refresh pattern for use throughout some or all of the differentrefresh cycles.

The memory refresh patterns may be encoded using one or more variablesstored in data storage 3308. For example, in embodiments where theplurality of memory segments are arranged in rows, each memory segmentidentifier may be configured to identify a particular location within arow of memory where a memory refresh should either begin or end. Forexample, in addition to Li and Hi, one or more additional variables maydefine which portions of the rows defined by Li and Hi are within thesegment.

FIG. 34 is an example flowchart of a process 3400 for determiningrefreshes for a memory chip (e.g., memory chip 2800 of FIG. 28). Process3100 may be implemented by software within a refresh controller (e.g.,refresh controller 3300 of FIG. 33) consistent with the presentdisclosure.

At step 3410, the refresh controller may store at least one memoryrefresh pattern to be implemented in refreshing a plurality of memorysegments included in each of a plurality of memory banks. For example,as explained above with respect to FIG. 33, the refresh pattern maycomprise a table including a plurality of memory segment identifiersassigned by the software to identify ranges of the plurality of memorysegments in a particular memory bank that are to be refreshed during arefresh cycle and ranges of the plurality of memory segments in theparticular memory bank that are not to be refreshed during the refreshcycle.

In some embodiments, the at least one refresh pattern may be encodedonto refresh controller (e.g., onto a read-only memory associated withor at least accessible by the refresh controller) during manufacture.Accordingly, the refresh controller may access the at least one memoryrefresh pattern but not store the same.

At steps 3420 and 3430, the refresh controller may use software toidentify which of the plurality of memory segments in a particularmemory bank are to be refreshed during a refresh cycle and which of theplurality of memory segments in the particular memory bank are not to berefreshed during the refresh cycle. For example, as explained above withrespect to FIG. 33, software executed by logic circuits (whetherconventional logic circuits such as CPUs and GPUs or processing groupson the same substrate as the memory chip, e.g., as depicted in FIG. 7A)may select the at least one memory refresh pattern. Moreover, therefresh controller may access the selected at least one memory refreshpattern to generate corresponding refresh signals during each refreshcycle. The refresh controller may refresh some or all portions withinthe defined segments that are not accessed during a current cycleaccording to the at least one memory refresh pattern and may skip otherportions of the defined segments that are set for access during thecurrent cycle.

At step 3440, the refresh controller may generate corresponding refreshcommands. For example, as depicted in FIG. 33, an adder 3305 maycomprise a logic circuit configured to nullify refresh signals forparticular segments that are not to be refreshed according to the atleast one memory refresh pattern in data storage 3307. Additionally oralternatively, a microprocessor (not shown in FIG. 33) may generateparticular refresh signals based on which segments are to be refreshedaccording to the at least one memory refresh pattern in data storage3307.

Method 3400 may further include additional steps. For example, inembodiments where the at least one memory refresh pattern is configuredto change (e.g., moving from L1, H1, and Inc1 to L2, H2, and Inc1 asshown in FIG. 33) every one, two, or other number of refresh cycles, therefresh controller may access a different portion of the data storagefor a next determination of refresh signals according to steps 3430 and3440. Similarly, if the software executed by logic circuits (whetherconventional logic circuits such as CPUs and GPUs or processing groupson the same substrate as the memory chip, e.g., as depicted in FIG. 7A)selects a new memory refresh pattern from the data storage for use inone or more future refresh cycles, the refresh controller may access adifferent portion of the data storage for a next determination ofrefresh signals according to steps 3430 and 3440.

When designing a memory chip and aiming for a certain capacity ofmemory, changes in memory capacity to a larger size or a smaller sizemay require redesign of the product and a redesign of a full mask set.Often, the product design is done in parallel with market research and,in some cases, the product design is completed before the marketresearch is available. Thus, there is the potential for disconnectsbetween product designs and actual demands of the market. The presentdisclosure proposes a way to flexibly provide a memory chips with memorycapacities commensurate with market demands. The design method mayinclude designing dies on a wafer along with appropriate interconnectcircuitry such that memory chips that may contain one or more dies canselectively be cut from the wafer in order to provide an opportunity toproduce memory chips of variable sized memory capacities from a singlewafer.

The present disclosure relates to systems and methods for fabricatingmemory chips by cutting them from a wafer. The method may be used forproducing selectable sized memory chips from the wafer. An exampleembodiment of a wafer 3501 containing dies 3503 is shown in FIG. 35A.Wafer 3501 may be formed from a semiconductor material (e.g., silicon(Si), silicon-germanium (SiGe), silicon on insulator (SOI), galliumnitride (GaN), aluminum nitride (AlN), aluminum gallium nitride (AlGaN),boron nitride (BN), gallium arsenide (GaAs), gallium aluminum arsenide(AlGaAs), indium nitride (InN) combination of thereof, and the like).Dies 3503 may include any suitable circuit elements (e.g., transistors,capacitors, resistors, and/or the like) which may include any suitablesemiconductor, dielectric or metallic components. Dies 3503 may beformed from a semiconductor material which may be the same or differentas the material of wafer 3501. In addition to dies 3503, wafer 3501 mayinclude other structures and/or circuitry. In some embodiments, one ormore coupling circuits may be provided and may couple together one ormore of the dies. In an example embodiment, such a coupling circuit mayinclude a bus shared by two or more dies 3503. Additionally, thecoupling circuit may include one or more logic circuits designed tocontrol circuitry associated with dies 3503 and/or to direct informationto/from dies 3503. In some cases, the coupling circuit may include amemory access management logic. Such logic may translate logical memoryaddresses into physical addresses associated with dies 3503. It shouldbe noted that the term fabrication, as used herein, may refercollectively to any of the steps for building the disclosed wafers,dies, and/or chips. For example, fabrication may refer to thesimultaneous laying out and forming of the various dies (and any othercircuitry) included on the wafer. Fabrication may also refer to thecutting of selectable sized memory chips from the wafer to include onedie, in some cases, or multiple dies in other cases. Of course, the termfabrication is not intended to be limited to these examples but mayinclude other aspects associated with generation of the disclosed memorychips and any or all of the intermediate structures.

Die 3503 or a group of dies may be used for fabrication of a memorychip. The memory chip may include a distributed processor, as describedin other sections of the present disclosure. As shown in FIG. 35B, die3503 may include a substrate 3507 and a memory array disposed on thesubstrate. The memory array may include one or more memory units, suchas, for example, memory banks 3511A-3511D designed to store data. Invarious embodiments, memory banks may include semiconductor-basedcircuit elements such as transistors, capacitors, and the like. In anexample embodiment, a memory bank may include multiple rows and columnsof storage units. In some cases, such a memory bank may have a capacitygreater than one megabyte. The memory banks may include dynamic orstatic access memory.

Die 3503 may further include a processing array disposed on thesubstrate, the processing array including a plurality of processorsubunits 3515A-3515D, as shown in FIG. 35B. As described above, eachmemory bank may include a dedicated processor subunit connected by adedicated bus. For example, processor subunit 3515A is associated withmemory bank 3511A via bus or connection 3512. It should be understoodthat various connections between memory banks 3511A-3511D and processorsubunits 3515A-3515D are possible, and only some illustrativeconnections are shown in FIG. 35B. In an example embodiment, processorsubunit may perform read/write operations for an associated memory bankand may further perform refreshing operations or any other suitableoperations relative to memory stored in the various memory banks.

As noted, die 3503 may include a first group of buses configured toconnect processor subunits with their corresponding memory banks. Anexample bus may include a set of wires or conductors that connectelectrical components and allow transfers of data and addresses to andfrom each memory bank and its associated processor subunit. In anexample embodiment, connection 3512 may serve as a dedicated bus forconnecting processor subunit 3515A to memory bank 3511A. Die 3503 mayinclude a group of such buses, each connecting a processor subunit to acorresponding, dedicated memory bank. Additionally, die 3503 may includeanother group of buses, each connecting processor subunits (e.g.,subunits 3515A-3515D) to each other. For example, such buses may includeconnections 3516A-3516D. In various embodiments data for memory banks3511A-3511D may be delivered via input-output bus 3530. In an exampleembodiment, input-output bus 3530 may carry data-related information,and command related information for controlling the operation of memoryunits of die 3503. Data information may include data for storing inmemory banks, data read from memory banks, processing results from oneor more of the processor subunits based on operations performed relativeto data stored in corresponding memory banks, command relatedinformation, various codes, etc.

In various cases, data and commands transmitted by input-output bus 3530may be controlled by input-output (IO) controller 3521. In an exampleembodiment, IO controller 3521 may control the flow of data from bus3530 to and from processor subunits 3515A-3515D. IO controller 3521 maydetermine from which one of processor subunits 3515A-3515D informationis retrieved. In various embodiments, IO controller 3521 may include afuse 3554 configured to deactivate IO controller 3521. Fuse 3554 may beused if multiple dies are combined together to form a larger memory chip(also referred to as a multi-die memory chip, as an alternative to asingle die memory chip that contains only one die). The multi-die memorychip may then use one of the IO controllers of one of the die unitsforming the multi-die memory chip while disabling other IO controllersrelated to the other die units by using fuses corresponding to the otherIO controllers.

As noted, each memory chip or predecessor die or group of dies mayinclude distributed processors associated with corresponding memorybanks. These distributed processors, in some embodiments, may bearranged in a processing array disposed on the same substrate as aplurality of memory banks Additionally, the processing array may includeone or more logic portions each including an address generator (alsoreferred to as address generator unit (AGU)). In some cases, the addressgenerator may be part of at least one processor subunit. The addressgenerator may generate memory addresses required for fetching data fromthe one or more memory banks associated with the memory chip.Address-generation calculations may involve integer arithmeticoperations, such as addition, subtraction, modulo operations, or bitshifts. The address generator may be configured to operate on multipleoperands at a time. Furthermore, multiple address generators may performmore than one address-calculation operation simultaneously. In variousembodiments, an address generator may be associated with a correspondingmemory bank. The address generators may be connected with theircorresponding memory banks by means of corresponding bus lines.

In various embodiments, a selectable sized memory chip may be formedfrom wafer 3501 by selectively cutting different regions of the wafer.As noted, the wafer may include a group of dies 3503, the groupincluding any group of two or more dies (e.g., 2, 3, 4, 5, 10, or moredies) included on the wafer. As will be discussed further below, in somecases, a single memory chip may be formed by cutting a portion of thewafer that includes just one of the dies of the group of dies. In suchcases, the resulting memory chip would include memory units associatedwith one die. In other cases, however, selectable sized memory chips maybe formed to include more than one die. Such memory chips may be formedby cutting regions of the wafer that include two or more dies of a groupof dies included on the wafer. In such cases, the dies together with acoupling circuit that couples together the dies provide a multi-diememory chip. Some additional circuit elements may also be wired on boardbetween chips, such as, for example, clock elements, data buses or anysuitable logic circuits.

In some cases, at least one controller associated with the group of diesmay be configured to control the operation of the group of dies as asingle memory chip (e.g., a multiple memory unit memory chip). Thecontroller may include one or more circuits that manage the flow of datagoing to and from the memory chip. A memory controller can be a part ofthe memory chip, or it can be a part of a separate chip not directlyrelated to the memory chip. In an example embodiment, the controller maybe configured to facilitate read and write requests or other commandsassociated with the distributed processors of the memory chip, and maybe configured to control any other suitable aspects of the memory chip(e.g., refreshing the memory chip, interacting with the distributedprocessors, etc.). In some cases, the controller may be part of die3503, and in other cases the controller may be laid out adjacent to die3503. In various embodiments, the controller may also include at leastone memory controller of at least one of the memory units included onthe memory chip. In some cases, a protocol used for accessinginformation on a memory chip may be agnostic to duplicate logic andmemory units (e.g., memory banks) that may be present on the memorychip. The protocol may be configured to have different IDs or addressranges for adequate access of data on the memory chip. An example of achip with such protocol may include a chip with a Joint Electron DeviceEngineering Council (JEDEC) double data rate (DDR) controller wheredifferent memory banks may have different address ranges, a serialperipheral interface (SPI) connection, where different memory units(e.g., memory banks) have different identifications (IDs), and the like.

In various embodiments, multiple regions may be cut from the wafer, withvarious regions including one or more dies. In some cases, each separateregion may be used to build a multi-die memory chip. In other cases,each region to be cut from the wafer may include a single die to providea single die memory chip. In some cases, two or more of the regions mayhave the same shape and have the same number of dies coupled to thecoupling circuit in the same way. Alternatively, in some exampleembodiments, a first group of regions may be used to form a first typeof the memory chip, and a second group of regions may be used to form asecond type of memory chip. For example, wafer 3501, as shown in FIG.35C may include a region 3505 that may include a single die, and asecond region 3504 may include a group of two dies. When region 3505 iscut from the wafer 3501, a single die memory chip will be provided. Whenregion 3504 is cut from the wafer 3501, a multi-die memory chip will beprovided. Groups shown in FIG. 35C are only illustrative, and variousother regions and groups of dies may be cut out from wafer 3501.

In various embodiments, dies may be formed on wafer 3501, such that theyare arranged along one or more rows of the wafer, as shown, for example,in FIG. 35C. The dies may share input-output bus 3530 corresponding toone or more rows. In an example embodiment, group of dies may be cut outfrom wafer 3501 using various cutting shapes where, when cutting out agroup of dies that may be used to form a memory chip, at least a portionof the shared input-output bus 3530 may be excluded (e.g., only aportion of input-output bus 3530 may be included as a part of the memorychip formed including a group of dies).

As previously discussed, when multiple dies (e.g., dies 3506A, and3506B, as shown in FIG. 35C) are used to form a memory chip 3517, one IOcontroller corresponding to one of the dies may be enabled andconfigured to control data flow to all the processor subunits of dies3506A and 3506B. For example, FIG. 35D shows memory dies 3506A and 3506Bcombined to form memory chip 3517 that includes memory banks3511A-3511H, processor subunits 3515A-3515H, IO controllers 3521A, and3521B and fuses 3554A and 3554B. It should be noted that memory chip3517 corresponds to a region 3517 of wafer 3501 prior to removal of thememory chip from the wafer. In other words, as used here and elsewherein the disclosure, regions 3504, 3505, 3517 etc. of wafer 3501 once cutfrom wafer 3501 will result in memory chips 3504, 3505, 3517, etc.Additionally, fuses herein are also referred to as disabling elements.In an example embodiment, fuse 3554B may be used to deactivate IOcontroller 3521B, and IO controller 3521A may be used to control dataflow to all memory banks 3511A-3511H by communicating data to processorsubunits 3515A-3515H. In an example embodiment, IO controller 3521A maybe connected to various processor subunits using any suitableconnection. In some embodiments, as further described below, processorsubunits 3515A-3515H may be interconnected, and IO controller 3521A maybe configured to control data flow to processor subunits 3515A-3515Hthat form processing logic of memory chip 3517.

In an example embodiment, IO controllers, such as controller 3521A and3521B and corresponding fuses 3554A and 3554B may be formed on wafer3501 together with the formation of memory banks 3511A-3511H andprocessor subunits 3515A-3515H. In various embodiments, when formingmemory chip 3517, one of the fuses (e.g., fuse 3554B) may be activatedsuch that dies 3506A and 3506B are configured to form memory chip 3517that functions as a single chip and is controlled by a singleinput-output controller (e.g., controller 3521A). In an exampleembodiment, activating a fuse may include applying a current to triggerthe fuse. In various embodiment, when more than one die is used forforming a memory chip, all but one IO controller may be deactivated viacorresponding fuses.

In various embodiments, as shown in FIG. 35C, multiple dies are formedon wafer 3501 together with a set of input-output buses, and/or controlbuses. An example input-output bus 3530 is shown in FIG. 35C. In anexample embodiment, one of the input-output buses (e.g., input-outputbus 3530) may be connected to multiple dies. FIG. 35C shows an exampleembodiment of input-output bus 3530 passing next to dies 3506A and3506B. Configuration of dies 3506A and 3506B and input-output bus 3530,as shown in FIG. 35C is only illustrative, and various otherconfigurations may be used. For example, FIG. 35E illustrates dies 3540formed on wafer 3501 and arranged in a hexagonal formation. A memorychip 3532 that includes four dies 3540 may be cut out form wafer 3501.In an example embodiment, memory chip 3532 may include a portion ofinput-output bus 3530 connected to the four dies by suitable bus lines(e.g., line 3533, as shown in FIG. 35E). In order to route informationto appropriate memory unit of memory chip 3532, memory chip 3532 mayinclude input/output controllers 3542A and 3542B placed at branch pointsfor input-output bus 3530. Controllers 3542A and 3542B may receivecommand data via input-output bus 3530 and select a branch of bus 3530for transmitting information to an appropriate memory unit. For example,if command data includes read/write information from/to memory unitsassociated with die 3546, controller 3542A may receive command requestand transmit data to a branch 3531A, of bus 3530, as shown in FIG. 35D,while controller 3542B may receive command request and transmit data toa branch 3531B. FIG. 35E indicates various cuts of different regionsthat may be made, where cut lines are represented by dashed lines.

In an example embodiment, a group of dies and interconnecting circuitrymay be designed for inclusion in a memory chip 3506 as shown in FIG.36A. Such an embodiment may include processor subunits (for in-memoryprocessing) that may be configured to communicate between one another.For example, each die to be included in memory chip 3506 may includevarious memory units such as memory banks 3511A-3511D, processorsubunits 3515A-3515D, and IO controllers 3521 and 3522. IO controllers3521 and 3522 may be connected in parallel to input-output bus 3530. IOcontroller 3521 may have a fuse 3554, and IO controller 3522 may have afuse 3555. In an example embodiment, processor subunits 3515A-3515D maybe connected by means of, for example, bus 3613. In some cases, one ofIO controller may be disabled using a corresponding fuse. For instance,IO controller 3522 may be disabled using fuse 3555, and IO controller3521 may control data flow into memory banks 3511A-3511D via processorsubunits 3515A-3515D connected to each other via bus 3613.

Configuration of memory units, as shown in FIG. 36A is onlyillustrative, and various other configurations may be formed by cuttingdifferent regions of wafer 3501. For example, FIG. 36B shows aconfiguration with three domains 3601-3603 containing memory units andconnected to input-output bus 3530. In an example embodiment, domains3601-3603 are connected to input-output bus 3530 using IO controlmodules 3521-3523 that may be disabled by corresponding fuses 3554-3556.Another example of embodiment of arranging domains containing memoryunits is shown in FIG. 36C where three domains 3601, 3602 and 3603 areconnected to input-output bus 3530 using bus lines 3611, 3612 and 3613.FIG. 36D shows, another example embodiment of memory chips 3506A-3506Dconnected to input-output buses 3530A and 3530B via IO controllers3521-3524. In an example embodiment, IO controllers may be deactivatedusing corresponding fuse elements 3554-3557, as shown in FIG. 36D.

FIG. 37 shows various groups of dies 3503, such as group 3713 and group3715 that may include one or more dies 3503. In an example embodiment,in addition to forming dies 3503 on wafer 3501, wafer 3501 may alsocontain logical circuits 3711 referred to as glue logic 3711. Glue logic3711 may take some space on wafer 3501 resulting in the fabrication of afewer number of dies per wafer 3501 as compared to a number of the diesthat could have been fabricated without the presence of glue logic 3711.However, the presence of glue logic 3711 may allow multiple dies to beconfigured to function together as a single memory chip. The glue logic,for example, may connect multiple dies, without having to makeconfiguration changes and without having to designate area inside any ofthe dies themselves for circuitry that is only used for connecting diestogether. In various embodiments, glue logic 3711 provides an interfacewith other memory controllers, such that multi-die memory chip functionsas a single memory chip. Glue logic 3711 may be cut together with agroup of dies as shown, for example, by group 3713. Alternatively, ifonly one die is required for the memory chip, as, for example, for group3715, glue logic may not be cut. For example, the glue logic may beselectively eliminated where not needed to enable cooperation betweendifferent dies. In FIG. 37, various cuts of different regions may bemade as shown, for example, by the dashed line regions. In variousembodiments, as shown in FIG. 37, one glue logic element 3711 may belaid out on the wafer for every two dies 3506. In some cases, one gluelogic element 3711 may be used for any suitable number of dies 3506forming a group of dies. Glue logic 3711 may be configured to beconnected to all the dies from the group of dies. In variousembodiments, dies connected to glue logic 3711 may be configured to forma multi-die memory chip and may be configured to form separate singledie memory chips when they are not connected to glue logic 3711. Invarious embodiments, dies connected to glue logic 3711 and designed tofunction together may be cut out from wafer 3501 as a group and mayinclude glue logic 3711 as indicated, for example, by group 3713. Thedies not connected to glue logic 3711 may be cut out from wafer 3501without including glue logic 3711 as indicated, for example, by group3715 to form a single die memory chip.

In some embodiments, during manufacturing of multi-die memory chips fromwafer 3501, one or more cutting shapes (e.g., shapes forming groups3713, 3715) may be determined for creating the desired set of themulti-die memory chips. In some cases, as shown by group 3715, thecutting shapes may exclude glue logic 3711.

In various embodiments, glue logic 3711 may be a controller forcontrolling multiple memory units of a multi-die memory chip. In somecases, glue logic 3711 may include parameters that may be modified byvarious other controllers. For example, a coupling circuit for multi-diememory chips may include a circuit for configuring parameters of gluelogic 3711 or parameters of memory controllers (e.g., processor subunits3515A-3515D, as shown, for example, in FIG. 35B). Glue logic 3711 may beconfigured to do a variety of tasks. For example, logic 3711 may beconfigured to determine which die may need to be addressed. In somecases, logic 3711 may be used to synchronize multiple memory units. Invarious embodiments, logic 3711 may be configured to control variousmemory units such that the memory units operate as a single chip. Insome cases, amplifiers between input-output bus (e.g., bus 3530, asshown in FIG. 35C) and processor subunits 3515A-3515D may be added toamplify a data signal from bus 3530.

In various embodiments, cutting complex shapes from wafer 3501 may betechnologically difficult/expensive, and a simpler cutting approach maybe adopted, provided that dies are aligned on wafer 3501. For example,FIG. 38A shows dies 3506 aligned to form a rectangular grid. In anexample embodiment, vertical cuts 3803 and horizontal cuts 3801 acrossentire wafer 3501 may be made to separate cut out groups of dies. In anexample embodiment, vertical and horizontal cuts 3803 and 3801 can leadto a group containing a selected number of dies. For instance, cuts 3803and 3801 can result in regions containing a single die (e.g., region3811A), regions containing two dies (e.g., region 3811B) and regionscontaining four dies (e.g., region 3811C). The regions formed by cuts3801 and 3803 are only illustrative, and any other suitable regions maybe formed. In various embodiments, depending on the alignment of dies,various cuts may be made. For instance, if dies are arranged in atriangular grid, as shown in FIG. 38B, cut lines such as lines 3802,3804, and 3806 may be used to make multi-die memory chips. For example,some regions may include six dies, five dies, four dies, three dies, twodies, one die, or any other suitable number of dies.

FIG. 38C shows bus lines 3530 arranged in a triangular grid, with dies3503 aligned in the centers of triangles formed by intersecting buslines 3530. Dies 3503 may be connected via bus lines 3820 to all theneighboring bus lines. By cutting a region containing two or moreadjacent dies (e.g., region 3822, as shown in FIG. 38C) at least one busline (e.g., line 3824) remains within region 3822, and bus line 3824 maybe used to supply data and commands to a multi-die memory chip formedusing region 3822.

FIG. 39 shows that various connections may be formed between processorsubunits 3515A-3515P to allow a group of memory units to act as a singlememory chip. For instance, a group 3901 of various memory units mayinclude a connection 3905 between processor subunit 3515B and subunit3515E. Connection 3905 may be used as a bus line for transmitting dataand commands to from subunit 3515B to subunit 3515E that can be used tocontrol a respective memory bank 3511E. In various embodiments,connections between processor subunits may be implemented during theformation of dies on wafer 3501. In some cases, additional connectionsmay be fabricated during a packaging stage of a memory chip formed fromseveral dies.

As shown in FIG. 39, processor subunits 3515A-3515P may be connected toeach other using various buses (e.g., connection 3905). Connection 3905may be free of timing hardware logic components such that data transfersbetween processor subunits and across connection 3905 may not becontrolled by timing hardware logic components. In various embodiments,buses connecting processor subunits 3515A-3515P may be laid out on wafer3501 prior to fabricating various circuits on wafer 3501.

In various embodiments, processor subunits (e.g., subunits 3515A-3515P)may be interconnected. For instance, subunits 3515A-3515P may beconnected by suitable buses (e.g., connections 3905). Connections 3905may be connect any one of subunits 3515A-3515P with any other of thesubunits 3515A-3515P. In an example embodiment, connected subunits maybe on a same die (e.g., subunits 3515A and 3515B) and in other cases,the connected subunits may be on different dies (e.g., subunits 3515Band 3515E). Connections 3905 may include dedicated buses for connectingsubunits and may be configured to efficiently transmit data betweensubunits 3515A-3515P.

Various aspects of the present disclosure relate to methods forproducing selectable sized memory chips from a wafer. In an exampleembodiment, selectable sized memory chips may be formed from one or moredies. The dies, as noted before, may be arranged along one or more rows,as shown, for example, in FIG. 35C. In some cases, at least one sharedinput-output bus corresponding to one or more rows may be laid out onwafer 3501. For example, bus 3530 may be laid out, as shown in FIG. 35C.In various embodiments, bus 3530 may be electrically connected to memoryunits of at least two of the dies, and the connected dies may be used toform a multi-die memory chip. In an example embodiment, one or morecontrollers (e.g., input-output controllers 3521 and 3522, as shown inFIG. 35B) may be configured to control the at memory units of least twodies that are used to form a multi-die memory chip. In variousembodiments, the dies with memory units connected to bus 3530 may be cutoff the wafer with at least one corresponding portion of the sharedinput-output bus (e.g., bus 3530, as shown in FIG. 35B) transmittinginformation to at least one controller (e.g., controllers 3521, 3522) toconfigure the controller to control the memory units of the connecteddies to function together as a single chip.

In some cases, the memory units located on wafer 3501 may be testedprior to manufacturing memory chips by cutting regions of wafer 3501.The testing may be done using at least one shared input-output bus(e.g., bus 3530, as shown in FIG. 35C). The memory chip may be formedfrom a group of dies containing memory units when the memory units passthe testing. The memory units that do not pass the testing may bediscarded, and not used for manufacturing of a memory chip.

FIG. 40 shows an example process 4000 of building memory chips from agroup of dies. At step 4011 of process 4000, the dies may be laid out onsemiconductor wafer 3501. At step 4015 the dies may be fabricated onwafer 3501 using any suitable approach. For example, dies may befabricated by etching wafer 3501, depositing various dielectric,metallic or semiconductor layers, and further etching of the depositedlayers, etc. For example, multiple layers may be deposited and etched.In various embodiments, layers may be n-type doped or p-type doped usingany suitable doping elements. For instance, semiconductor layers may ben-type doped with phosphorus and may be p-type doped with boron. Dies3503, as shown in FIG. 35A may be separated from each other by a spacethat may be used to cut dies 3503 out of wafer 3501. For example, dies3503 may be spaced apart from each other by spacing regions, where thewidth of the spacing regions may be selected to allow wafer cuts in thespacing regions.

At step 4017, dies 3503 may be cut out from wafer 3501 using anysuitable approach. In an example embodiment dies 3503 may be cut outusing a laser. In an example embodiment, wafer 3501 may be scribed firstfollowing by mechanical dicing. Alternatively, mechanical dicing saw maybe used. In some cases, a stealth dicing process may be used. Duringdicing, wafer 3501 may be mounted on a dicing tape for holding dies oncethey are cut out. In various embodiments large cuts may be done, asshown for example in FIG. 38A, by cuts 3801 and 3803 or in FIG. 38B asshown by cuts 3802, 3804, or 3806. Once dies 3503 are cut outindividually or in groups, as shown for example by group 3504 in FIG.35C, dies 3503 may be packaged. Packaging of dies may include formingcontacts to dies 3503, depositing protective layers over contacts,attaching heat managing devices (e.g., heatsinks) and encapsulating dies3503. In various embodiments, depending on how many dies are selected toform a memory chip, appropriate configuration of contacts and buses maybe used. In an example embodiment, some of the contacts betweendifferent dies forming the memory chip may be made during memory chippackaging.

FIG. 41A shows an example process 4100 for manufacturing memory chipscontaining multiple dies. Step 4011 of process 4100 may be the same asstep 4011 of process 4000. At step 4111, glue logic 3711, as shown inFIG. 37 may laid out on wafer 3501. Glue logic 3711 may be any suitablelogic for controlling operations of dies 3506, as shown in FIG. 37. Asdescribed before, the presence of glue logic 3711 may allow multipledies to function as a single memory chip. Glue logic 3711 may provide aninterface with other memory controllers, such that memory chip formedfrom multiple dies functions as a single memory chip.

At step 4113 of process 4100, buses (e.g., input-output buses andcontrol buses) may be laid out on wafer 3501. The buses may be laid outsuch that they are connected with various dies and logic circuits, suchas glue logic 3711. In some cases, buses may connect memory units. Forexample, buses may be configured to connect processor subunits ofdifferent dies. At step 4115, dies, glue logic and buses may befabricated using any suitable approach. For example, logic elements maybe fabricated by etching wafer 3501, depositing various dielectric,metallic or semiconductor layers, and further etching of the depositedlayers, etc. Buses may be fabricated using, for example, metalevaporation.

At step 4140, cutting shapes may be used to cut groups of dies connectedto a single glue logic 3711, as shown, for example, in FIG. 37. Cuttingshapes may be determined using memory requirements for a memory chipcontaining multiple dies 3503. For instance, FIG. 41B shows a process4101, which may be a variant of process 4100, where step 4140 of process4100 may be preceded by steps 4117 and 4119. At step 4117 a system forcutting wafer 3501 may receive instructions describing requirements fora memory chip. For example, requirements may include forming a memorychip including four dies 3503. In some cases, a program software maydetermine a periodic pattern for group of dies and glue logic 3711 atstep 4119. For instance, a periodic pattern may include two glue logic3711 elements and four dies 3503 with every two dies connected to oneglue logic 3711. Alternatively, at step 4119 the pattern may be providedby a designer of memory chips.

In some cases, the pattern may be selected to maximize a yield of memorychips from wafer 3501. In an example embodiment, memory units of dies3503 may be tested to identify dies with faulty memory units (such diesare referred to as faulty of failed dies), and based on the location offaulty dies, groups of dies 3503 that contain memory units that pass thetest can be identified and an appropriate cutting pattern can bedetermined. For example, if a large number of dies 3503 fail at edges ofwafer 3501, a cutting pattern may be determined to avoid dies at theedges of wafer 3501. Other steps of process 4101, such as steps 4011,4111, 4113, 4115, and 4140 may be the same as the same numbered steps ofprocess 4100.

FIG. 41C shows an example process 4102 that may be a variation ofprocess 4101. Steps 4011, 4111, 4113, 4115, and 4140 of process 4102 maybe the same as the same numbered steps of process 4101, step 4131 ofprocess 4102 may substitute step 4117 of process 4101, and step 4133 ofprocess 4102 may substitute step 4119 of process 4101. At step 4131, asystem for cutting wafer 3501 may receive instructions describingrequirements for a first set of memory chips and a second set of memorychips. For example, requirements may include forming the first set ofmemory chip with memory chips consisting of four dies 3503, and forminga second set of memory chip with memory chips consisting of two dies3503. In some cases, more than two sets of memory chips may need to beformed from wafer 3501. For instance, a third set of memory chips mayinclude memory chips consisting of only one die 3503. In some cases, atstep 4133, a program software may determine a periodic pattern for agroup of dies and glue logic 3711 for forming memory chips for each setof memory chips. For instance, a first set of memory chips may includememory chips containing two glue logic 3711 and four dies 3503 withevery two dies connected to one glue logic 3711. In various embodiments,glue logic units 3711 for the same memory chip may be linked together toact as a single glue logic. For example, during fabrication of gluelogic 3711 appropriate bus lines may be formed linking glue logic units3711 with one another.

The second set of memory chips may include memory chips containing oneglue logic 3711 and two dies 3503 with dies 3503 connected to glue logic3711. In some cases, when a third set of memory chips is selected, andwhen it includes a memory chip consisting of a single die 3503, no gluelogic 3711 may be needed for these memory chips.

When designing memory chips or memory instances within a chip, oneimportant characteristic is the number of words that can be accessedsimultaneously during a single clock cycle. The more addresses (e.g.,addresses along rows, also called words or word lines, and columns, alsocalled bits or bitlines) that can be accessed at the same time forreading and/or writing, the faster the memory chip. While there has beensome activity in developing memories that include multi-way ports thatallow access to multiple addresses at the same time, e.g., for buildingregister files, cashes, or shared memories, most instances use a memorymat that is larger in size and that supports the multiple addressaccesses. However, DRAM chips usually include a single bit line and asingle row line connected to each capacitor of each memory cell.Accordingly, embodiments of the present disclosure seek to providemulti-port access on existing DRAM chips without modifying thisconventional single-port memory structure of DRAM arrays.

Embodiments of the present disclosure may clock memory instances orchips at twice the speed of logic circuits using the memory. Any logiccircuits using the memory may therefore “correspond” to the memory andany components thereof. Accordingly, embodiments of the presentdisclosure may retrieve or write to two addresses in two memory arrayclock cycles, which are equivalent to a single processing clock cyclefor the logic circuits. The logic circuits may comprise circuits such ascontrollers, accelerators, GPUs, or CPUs or may comprise processinggroups on the same substrate as the memory chip, e.g., as depicted inFIG. 7A. As explained above with respect to FIG. 3A, a “processinggroup” may refer to two or more processor subunits and theircorresponding memory banks on a substrate. The group may represent aspatial distribution on the substrate and/or a logical grouping for thepurposes of compiling code for execution on memory chip 2800.Accordingly, as described above with respect to FIG. 7A, a substratewith the memory chip may include a memory array with a plurality ofbanks, such as banks 2801 a and other banks shown in FIG. 28.Furthermore, the substrate may also include a processing array that mayinclude a plurality of processor subunits (such as subunits 730 a, 730b, 730 c, 730 d, 730 e, 730 f, 730 g, and 730 h shown in FIG. 7A).

Accordingly, embodiments of the present disclosure may retrieve datafrom the array at each one of two consecutive memory cycles in orderhandle two addresses for each logic cycle and provide the logic with tworesults as though the single-port memory array were a two-port memorychip. Additional clocking may allow for memory chips of the presentdisclosure to function as though the single-port arrays are a two-portmemory instance, a three-port memory instance, a four-port memoryinstance, or any other multi-port memory instance.

FIG. 42 depicts example circuitry 4200 providing dual-port access alongcolumns of a memory chip in which circuitry 4200 is used, consistentwith the present disclosure. The embodiment depicted in FIG. 42 may useone memory array 4201 with two column multiplexers (“muxes”) 4205 a and4205 b to access two words on the same row during a same clock cycle fora logic circuit. For example, during a memory clock cycle, RowAddrA isused in row decoder 4203, and ColAddrA is used in multiplexer 4205 a tobuffer data from a memory cell with address (RowAddrA, ColAddrA). Duringthe same memory clock cycle, ColAddrB is used in multiplexer 4205 b tobuffer data from a memory cell with address (RowAddrA, ColAddrB). Thus,circuitry 4200 may allow for dual-port access to data (e.g., DataA andDataB) stored on memory cells at two different addresses along the samerow or word line. Thus, the two addresses may share a row such that therow decoder 4203 activates the same word line for both retrievals.Moreover, embodiments like the example depicted in FIG. 42 may usecolumn muxes such that two addresses may be accessed during a samememory clock cycle.

Similarly, FIG. 43 depicts example circuitry 4300 providing dual-portaccess along rows of a memory chip in which circuitry 4300 is used,consistent with the present disclosure. The embodiment depicted in FIG.43 may use one memory array 4301 with a row decoder 4303 coupled with amultiplexer (“mux”) to access two words on the same column during a sameclock cycle for a logic circuit. For example, on the first of two memoryclock cycles, RowAddrA is used in row decoder 4303, and ColAddrA is usedin column multiplexer 4305 to buffer data (e.g., to the “Buffered Word”buffer of FIG. 43) from a memory cell with address (RowAddrA, ColAddrA).On the second of two memory clock cycles, RowAddrB is used in rowdecoder 4303, and ColAddrA is used in column multiplexer 4305 to bufferdata from a memory cell with address (RowAddrB, ColAddrA). Thus,circuitry 4300 may allow for dual-port access to data (e.g., DataA andDataB) stored on memory cells at two different addresses along the samecolumn or bitline. Thus, the two addresses may share a row such that thecolumn decoder (which may be separate from or combined with one or morecolumn multiplexers, as depicted in FIG. 43) activates the same bitlinefor both retrievals. Embodiments like the example depicted in FIG. 43may use two memory clock cycles because row decoder 4303 may need onememory clock cycle to activate each word line. Accordingly, a memorychip using circuitry 4300 may function as a dual-port memory if clockedat least twice as fast as a corresponding logic circuit.

Accordingly, as explained above, FIG. 43 may retrieve DataA and DataBduring two memory clock cycles, which is faster than a clock cycle for acorresponding logic circuit. For example, the row decoder (e.g., rowdecoder 4303 of FIG. 43) and the column decoder (which may be separatefrom or combined with one or more column multiplexers, as depicted inFIG. 43) may be configured to be clocked at a rate at least twice a rateof a corresponding logic circuit generating the two addresses. Forexample, a clock circuit for circuitry 4300 (not shown in FIG. 43) mayclock circuitry 4300 according to a rate at least twice a rate of acorresponding logic circuit generating the two addresses.

The embodiment of FIGS. 42 and 43 may be used separately or combined.Accordingly, circuitry (e.g., circuitry 4200 or 4300) providingdual-port functionality on a single-port memory array or mat maycomprise a plurality of memory banks arranged along at least one row andat least one column. The plurality of memory banks are depicted asmemory array 4201 in FIG. 42 and as memory array 4301 in FIG. 43. Theembodiments may further use at least one row multiplexer (as depicted inFIG. 43) or at least one column multiplexer (as depicted in FIG. 42)configured to receive, during a single clock cycle, two addresses forreading or writing. Moreover, the embodiments may use a row decoder(e.g., row decoder 4203 of FIG. 42 and row decoder 4303 of FIG. 43) anda column decoder (which may be separate from or combined with one ormore column multiplexers, as depicted in FIGS. 42 and 43) to read fromor write to the two addresses. For example, the row decoder and columndecoder may, during a first cycle, retrieve a first of the two addressesfrom the at least one row multiplexer or the at least one columnmultiplexer and decode a word line and a bitline corresponding to thefirst address. Moreover, the row decoder and column decoder may, duringa second cycle, retrieve a second of the two addresses from the at leastone row multiplexer or the at least one column multiplexer and decode aword line and a bitline corresponding to the second address. Theretrievals may each comprise activating a word line corresponding to anaddress using the row decoder and activating a bit line on the activatedword line corresponding to the address using the column decoder.

Although described above for retrievals, the embodiments of FIGS. 42 and43, whether implemented separately or in combination, may include writecommands. For example, during the first cycle, the row decoder andcolumn decoder may write first data retrieved from the at least one rowmultiplexer or the at least one column multiplexer to the first of thetwo addresses. Moreover, during the second cycle, the row decoder andcolumn decoder may write second data retrieved from the at least one rowmultiplexer or the at least one column multiplexer to the second of thetwo addresses.

The example of FIG. 42 shows this process when the first and secondaddresses share a word line address while the example of FIG. 43 showsthis process when the first and second addresses share a column address.As described further with respect to FIG. 47 below, the same process maybe implemented when the first and second address do not share either aword line address or a column address.

Accordingly, although the examples above provide dual-port access alongat least one of rows or columns, additional embodiments may providedual-port access along both rows and columns. FIG. 44 depicts examplecircuitry 4400 providing dual-port access along both rows and columns ofa memory chip in which circuitry 4400 is used, consistent with thepresent disclosure. Accordingly, circuitry 4700 may represent acombination of circuitry 4200 of FIG. 42 with circuitry 4300 of FIG. 43.

The embodiment depicted in FIG. 44 may use one memory array 4401 with arow decoder 4403 coupled with a multiplexer (“mux”) to access two rowsduring a same clock cycle for a logic circuit. Moreover, the embodimentdepicted in FIG. 44 may use memory array 4401 with a column decoder (ormultiplexer) 4405 coupled with a multiplexer (“mux”) to access twocolumns during the same clock cycle. For example, on the first of twomemory clock cycles, RowAddrA is used in row decoder 4403, and ColAddrAis used in column multiplexer 4405 to buffer data (e.g., to the“Buffered Word” buffer of FIG. 44) from a memory cell with address(RowAddrA, ColAddrA). On the second of two memory clock cycles, RowAddrBis used in row decoder 4403, and ColAddrB is used in column multiplexer4405 to buffer data from a memory cell with address (RowAddrB,ColAddrB). Thus, circuitry 4400 may allow for dual-port access to data(e.g., DataA and DataB) stored on memory cells at two differentaddresses. Embodiments like the example depicted in FIG. 44 may use theadditional buffer because row decoder 4403 may need one memory clockcycle to activate each word line. Accordingly, a memory chip usingcircuitry 4400 may function as a dual-port memory if clocked at leasttwice as fast as a corresponding logic circuit.

Although not depicted in FIG. 44, circuitry 4400 may further include theadditional circuitry of FIG. 46 (described further below) along the rowsor word lines and/or similar additional circuitry along the columns orbitlines. Accordingly, circuitry 4400 may activate correspondingcircuitry (e.g., by opening one or more switching elements, such as oneor more of switching elements 4613 a, 4613 b, and the like of FIG. 46)to activate disconnected portions including the addresses (e.g., byconnecting voltages or allowing current to flow to the disconnectedportions). Accordingly, the circuitry may “correspond” when elements ofthe circuitry (such as lines or the like) include locations identifiedthe addresses and/or when elements of the circuitry (such as theswitching elements) control a supply or voltage and/or a flow of currentto memory cells identified by the addresses. Circuitry 4400 may then userow decoder 4403 and column multiplexer 4405 to decode correspondingword lines and bitlines to retrieve data from or write data to theaddresses, which are located in the activated disconnected portions.

As further depicted in FIG. 44, circuitry 4400 may further use at leastone row multiplexer (depicted separate from row decoder 4403 but may beincorporate therein) and/or at least one column multiplexer (e.g.,depicted separate from column multiplexer 4405 but may be incorporatetherein) configured to receive, during a single clock cycle, twoaddresses for reading or writing. Accordingly, the embodiments may use arow decoder (e.g., row decoder 4403) and a column decoder (which may beseparate from or combined with column multiplexer 4405) to read from orwrite to the two addresses. For example, the row decoder and columndecoder may, during a memory clock cycle, retrieve a first of the twoaddresses from the at least one row multiplexer or the at least onecolumn multiplexer and decode a word line and a bitline corresponding tothe first address. Moreover, the row decoder and column decoder may,during the same memory cycle, retrieve a second of the two addressesfrom the at least one row multiplexer or the at least one columnmultiplexer and decode a word line and a bitline corresponding to thesecond address.

FIGS. 45A and 45B depict existing duplication techniques for providingdual-port functionality on a single-port memory array or mat. As shownin FIG. 45A, dual-port reading may be provided by keeping duplicatecopies of data in sync across memory arrays or mats. Accordingly,reading may be performed from both copies of the memory instance, asdepicted in FIG. 45A. Moreover, as shown in FIG. 45B, dual-port writingmay be provided by duplicating all writes across the memory arrays ormats. For example, the memory chip may require that logic circuits usingthe memory chip send write commands in duplicate, one for each duplicatecopy of the data. Alternatively, in some embodiments, as shown in FIG.45A, additional circuitry may allow for the logic circuits using thememory instance to send single write commands that are automaticallyduplicated by the additional circuitry to generate duplicate copies ofthe written data across the memory arrays or mats in order to keep thecopies in sync. The embodiments of FIGS. 42, 43, and 44 may reduce theredundancy from these existing duplication techniques either by usingmultiplexers to access two bitlines in a single memory clock cycle(e.g., as depicted in FIG. 42) and/or by clocking the memory faster thana corresponding logic circuit (e.g., as depicted in FIGS. 43 and 44) andproviding additional multiplexers to handle additional addresses ratherthan duplicating all data in the memory.

In addition to the faster clocking and/or additional multiplexersdescribed above, embodiments of the present disclosure may use circuitrythat disconnects the bitlines and/or word lines at some points withinthe memory array. Such embodiments may allow for multiple simultaneousaccess to the array as long as the row and column decoders accessdifferent locations that are not coupled to the same portions of thedisconnect circuitry. For example, locations with different word linesand bitlines may be accessed simultaneously because the disconnectingcircuitry may allow the row and column decodes to access the differentaddresses without electrical interference. The granularity of thedisconnected regions within the memory array may be weighed against theadditional area required by the disconnect circuitry during design ofthe memory chip.

An architecture for implementing such simultaneous access is depicted inFIG. 46. In particular, FIG. 46 depicts example circuitry 4600 providingdual-port functionality on a single-port memory array or mat. Asdepicted in FIG. 46, circuitry 4600 may include a plurality of memorymats (e.g., memory mat 4609 a, mat 4609 b, and the like) arranged alongat least one row and at least one column. The layout of circuitry 4600further includes a plurality of word lines, such as word lines 4611 aand 4611 b corresponding to rows and bitlines 4615 a and 4615 bcorresponding to columns.

The example of FIG. 46 includes twelve memory mats, each with two linesand eight columns. In other embodiments, the substrate may include anynumber of memory mats, and each memory mat may include any number oflines and any number of columns Some memory mats may include a samenumber of lines and columns (as shown in FIG. 46) while other memorymats may include different numbers of lines and/or columns.

Although not depicted in FIG. 46, circuitry 4600 may further use atleast one row multiplexer (either separate from or incorporated with rowdecoder 4601 a and/or 4601 b) or at least one column multiplexer (e.g.,column multiplexer 4603 a and/or 4603 b) configured to receive, during asingle clock cycle, two (or three or any plurality of) addresses forreading or writing. Moreover, the embodiments may use a row decoder(e.g., row decoder 4601 a and/or 4601 b) and a column decoder (which maybe separate from or combined with column multiplexer 4603 a and/or 4603b) to read from or write to the two (or more) addresses. For example,the row decoder and column decoder may, during a memory clock cycle,retrieve a first of the two addresses from the at least one rowmultiplexer or the at least one column multiplexer and decode a wordline and a bitline corresponding to the first address. Moreover, the rowdecoder and column decoder may, during the same memory cycle, retrieve asecond of the two addresses from the at least one row multiplexer or theat least one column multiplexer and decode a word line and a bitlinecorresponding to the second address. As explained above, as long as thetwo addresses are in different locations that are not coupled to thesame portions of the disconnect circuitry (e.g., switching elements suchas 4613 a, 4613 b, and the like), the access may occur during the samememory clock cycle. Additionally, circuitry 4600 may access a first twoaddresses simultaneously during a first memory clock cycle and then asecond two addresses simultaneously during a second memory clock cycle.In such embodiments, a memory chip using circuitry 4600 may function asa four-port memory if clocked at least twice as fast as a correspondinglogic circuit.

FIG. 46 further includes at least one row circuit and at least onecolumn circuit configured to function as switches. For example,corresponding switching elements such as 4613 a, 4613 b, and the likemay comprise transistors or any other electrical element configured toallow or stop current to flow and/or connect or disconnect voltages fromthe word line or bitline connected to switching elements such as 4613 a,4613 b, and the like. Thus, the corresponding switching elements maydivide circuitry 4600 into disconnected portions. Although depicted ascomprising single rows and sixteen columns of each row, the disconnectedregions within the circuitry 4600 may include differing levels ofgranularity depending on design of the circuitry 4600.

Circuitry 4600 may use a controller (e.g., row control 4607) to activatecorresponding ones of the at least one row circuit and the at least onecolumn circuit in order to activate corresponding disconnected regionsduring the address operations described above. For example, circuitry4600 may transmit one or more control signals to close correspondingones of the switching elements (e.g., switching elements 4613 a, 4613 b,and the like). In embodiments where switching elements 4613 a, 4613 b,and the like comprises transistors, the control signals may comprisevoltages to open the transistors.

Depending on the disconnected regions including the addresses, more thanone of the switching elements may be activated by circuitry 4600. Forexample, to reach an address within memory mat 4609 b of FIG. 46, theswitching element allowing access to memory mat 4609 a must be opened aswell as the switching element allowing access to memory mat 4609 b. Rowcontrol 4607 may determine the switching elements to activate in orderto retrieve a particular address within circuitry 4600 according to theparticular address.

FIG. 46 represents an example of circuitry 4600 used to divide wordlines of a memory array (e.g., comprising memory mat 4609 a, mat 4609 b,and the like). However, other embodiments may use similar circuitry(e.g., switching elements dividing memory chip 4600 into disconnectedregions) to divide bitlines of the memory array. Accordingly, thearchitecture of circuitry 4600 may be used in dual-column access likethat depicted in FIG. 42 or FIG. 44 as well as dual-row access like thatdepicted in FIG. 43 or FIG. 44.

A process for multi-cycle access to memory arrays or mats is depicted inFIG. 47A. In particular, FIG. 47A is an example flowchart of a process4700 for providing dual-port access on a single-port memory array or mat(e.g., using circuitry 4300 of FIG. 43 or circuitry 4400 of FIG. 44)Process 4700 may be executed using row and column decoders consistentwith the present disclosure, such as row decoder 4303 or 4403 of FIG. 43or 44, respectively, and a column decoder (which may be separate from orcombined with one or more column multiplexers, such as columnmultiplexer 4305 or 4405 depicted in FIG. 43 or 44, respectively).

At step 4710, during a first memory clock cycle, the circuitry may useat least one row multiplexer and at least one column multiplexer todecode a word line and a bitline corresponding to a first of twoaddresses. For example, the at least one row decoder may activate a wordline, and the at least one column multiplexer may amplify a voltage froma memory cell along the activated word line and corresponding to thefirst address. The amplified voltage may be provided to a logic circuitusing a memory chip including the circuitry or buffered according tostep 4720 described below. The logic circuits may comprise circuits suchas GPUs or CPUs or may comprise processing groups on the same substrateas the memory chip, e.g., as depicted in FIG. 7A.

Although described above as a read operation, method 4700 may similarlyprocess a write operation. For example, the at least one row decoder mayactivate a word line, and the at least one column multiplexer may applya voltage to a memory cell along the activated word line andcorresponding to the first address to write new data to the memory cell.In some embodiments, the circuitry may provide confirmation of the writeto the logic circuit using the memory chip including the circuitry orbuffer the confirmation according to step 4720 below.

At step 4720, the circuitry may buffer the retrieved data of the firstaddress. For example, as depicted in FIGS. 43 and 44, the buffer mayallow the circuitry to retrieve a second of the two addresses (asdescribed in step 4730 below) and return the results of both retrievalstogether. The buffer may comprise a register, an SRAM, a nonvolatilememory, or any other data storage device.

At step 4730, during a second memory clock cycle, the circuitry may usethe at least one row multiplexer and the at least one column multiplexerto decode a word line and a bitline corresponding to a second address ofthe two addresses. For example, the at least one row decoder mayactivate a word line, and the at least one column multiplexer mayamplify a voltage from a memory cell along the activated word line andcorresponding to the second address. The amplified voltage may beprovided to a logic circuit using a memory chip including the circuitry,whether individually or together with a buffered voltage, e.g., fromstep 4720. The logic circuits may comprise circuits such as GPUs or CPUsor may comprise processing groups on the same substrate as the memorychip, e.g., as depicted in FIG. 7A.

Although described above as a read operation, method 4700 may similarlyprocess a write operation. For example, the at least one row decoder mayactivate a word line, and the at least one column multiplexer may applya voltage to a memory cell along the activated word line andcorresponding to the second address to write new data to the memorycell. In some embodiments, the circuitry may provide confirmation of thewrite to the logic circuit using the memory chip including thecircuitry, whether individually or together with a buffered voltage,e.g., from step 4720.

At step 4740, the circuitry may output the retrieved data of the secondaddress with the buffered first address. For example, as depicted inFIGS. 43 and 44, the circuitry may return the results of both retrievals(e.g., from steps 4710 and 4730) together. The circuitry may return theresults to a logic circuit using a memory chip including the circuitry,The logic circuits may comprise circuits such as GPUs or CPUs or maycomprise processing groups on the same substrate as the memory chip,e.g., as depicted in FIG. 7A.

Although described with reference to multiple cycles, if the twoaddresses share a word lines, as depicted in FIG. 42, method 4700 mayallow for single-cycle access to the two addresses. For example, steps4710 and 4730 may occur during a same memory clock cycle since multiplecolumn multiplexers may decode different bitlines on a same word lineduring the same memory clock cycle. In such embodiments, the bufferingstep 4720 may be skipped.

A process for simultaneous access (e.g., using circuitry 4600 describedabove) is depicted in FIG. 47B. Accordingly, although shown in sequence,the steps of FIG. 47B may all occur during a same memory clock cycle,and at least some steps (e.g., steps 4760 and 4780 or steps 4770 and4790) may be executed simultaneously. In particular, FIG. 47B is anexample flowchart of a process 4750 for providing dual-port access on asingle-port memory array or mat (e.g., using circuitry 4200 of FIG. 42or circuitry 4600 of FIG. 46) Process 4750 may be executed using row andcolumn decoders consistent with the present disclosure, such as rowdecoder 4203 or rows decoders 4601 a and 4601 b of FIG. 42 or 46,respectively, and a column decoder (which may be separate from orcombined with one or more column multiplexers, such as columnmultiplexers 4205 a and 4205 b or column multiplexers 4603 a and 4306 bdepicted in FIG. 42 or 46, respectively).

At step 4760, during a memory clock cycle, the circuitry may activatecorresponding ones of at least one row circuit and at least one columncircuit based on a first of two addresses. For example, the circuitrymay transmit one or more control signals to close corresponding ones ofswitching elements comprising the at least one row circuit and the atleast one column circuit. Accordingly, the circuitry may access acorresponding disconnected region including the first of the twoaddresses.

At step 4770, during the memory clock cycle, the circuitry may use atleast one row multiplexer and at least one column multiplexer to decodea word line and a bitline corresponding to the first address. Forexample, the at least one row decoder may activate a word line, and theat least one column multiplexer may amplify a voltage from a memory cellalong the activated word line and corresponding to the first address.The amplified voltage may be provided to a logic circuit using a memorychip including the circuitry. For example, as described above, the logiccircuits may comprise circuits such as GPUs or CPUs or may compriseprocessing groups on the same substrate as the memory chip, e.g., asdepicted in FIG. 7A.

Although described above as a read operation, method 4500 may similarlyprocess a write operation. For example, the at least one row decoder mayactivate a word line, and the at least one column multiplexer may applya voltage to a memory cell along the activated word line andcorresponding to the first address to write new data to the memory cell.In some embodiments, the circuitry may provide confirmation of the writeto the logic circuit using the memory chip including the circuitry.

At step 4780, during the same cycle, the circuitry may activatecorresponding ones of the at least one row circuit and the at least onecolumn circuit based on a second of the two addresses. For example, thecircuitry may transmit one or more control signals to closecorresponding ones of switching elements comprising the at least one rowcircuit and the at least one column circuit. Accordingly, the circuitrymay access a corresponding disconnected region including the second ofthe two addresses.

At step 4790, during the same cycle, the circuitry may use the at leastone row multiplexer and the at least one column multiplexer to decode aword line and a bitline corresponding to the second address. Forexample, the at least one row decoder may activate a word line, and theat least one column multiplexer may amplify a voltage from a memory cellalong the activated word line and corresponding to the second address.The amplified voltage may be provided to a logic circuit using thememory chip including the circuitry. For example, as described above,the logic circuits may comprise conventional circuits such as GPUs orCPUs or may comprise processing groups on the same substrate as thememory chip, e.g., as depicted in FIG. 7A.

Although described above as a read operation, method 4500 may similarlyprocess a write operation. For example, the at least one row decoder mayactivate a word line, and the at least one column multiplexer may applya voltage to a memory cell along the activated word line andcorresponding to the second address to write new data to the memorycell. In some embodiments, the circuitry may provide confirmation of thewrite to the logic circuit using the memory chip including thecircuitry.

Although described with reference to a single cycle, if the twoaddresses are in disconnected regions sharing word lines or bitlines (orotherwise sharing switching elements in the at least one row circuit andthe at least one column circuit), method 4500 may allow for multi-cycleaccess to the two addresses. For example, steps 4760 and 4770 may occurduring a first memory clock cycle in which a first row decoder and afirst column multiplexer may decode the word line and bitlinecorresponding to the first address while steps 4780 and 4790 may occurduring a second memory clock cycle in which a second row decoder and asecond column multiplexer may decode the word line and bitlinecorresponding to the second address.

A further example of architecture for dual-port access along both rowsand columns is depicted in FIG. 48. In particular, FIG. 48 depictsexample circuitry 4800 providing dual-port access along both rows andcolumns using multiple row decoders in combination with multiple columnmultiplexers. In FIG. 48, row decoder 4801 a may access a first wordline, and column multiplexer 4803 a may decode data from one or morememory cells along the first word line while row decoder 4801 b mayaccess a second word line, and column multiplexer 4803 b may decode datafrom one or more memory cells along the second word line.

As described with respect to FIG. 47B, this access may be simultaneousduring one memory clock cycles. Accordingly, similar to the architectureof FIG. 46, the architecture of FIG. 48 (including the memory matsdescribed in FIG. 49 below) may allow for multiple addresses to beaccessed in a same clock cycle. For example, the architecture of FIG. 48may include any number of row decoders and any number of columnmultiplexers such that a number of addresses corresponding to the numberof row decoder and column multiplexers may be accessed all within asingle memory clock cycle.

In other embodiments, this access may be sequential along two memoryclock cycles. By clocking memory chip 4800 faster than a correspondinglogic circuit, two memory clock cycles may be equivalent to one clockcycle for the logic circuit using the memory. For example, as describedabove, the logic circuits may comprise conventional circuits such asGPUs or CPUs or may comprise processing groups on the same substrate asthe memory chip, e.g., as depicted in FIG. 7A.

Other embodiments may allow for simultaneous access. For example, asdescribed with respect to FIG. 42, multiple column decoders (which maycomprise column multiplexers such as 4803 a and 4803 b as shown in FIG.48) may read multiple bitlines along a same word line during a singlememory clock cycle. Additionally or alternatively, as described withrespect to FIG. 46, circuitry 4800 may incorporate additional circuitrysuch that this access may be simultaneous. For example, row decoder 4801a may access a first word line, and column multiplexer 4803 a may decodedata from a memory cell along the first word line during a same memoryclock cycle in which row decoder 4801 b accesses a second word line, andcolumn multiplexer 4803 b decodes data from a memory cell along thesecond word line.

The architecture of FIG. 48 may be used with modified memory matsforming the memory banks as shown in FIG. 49. In FIG. 49, each memorycell (depicted as a capacitor similar to DRAM but may also comprise anumber of transistors arranged in a manner similar to SRAM or any othermemory cell) is accessed by two word lines and by two bit lines.Accordingly, memory mat 4900 of FIG. 49 allows for access of twodifferent bits simultaneously or even access to a same bit by twodifferent logic circuits. However, the embodiment of FIG. 49 uses amodification to the memory mats rather than implementing a dual-portsolution on standard DRAM memory mats, which are wired for single-portaccess, as the embodiments above do.

Although described with two ports, any of the embodiments describedabove may be extended to more than two ports. For example, theembodiments of FIGS. 42, 46, 48, and 49 may include additional column orrow multiplexers, respectively, to provide access to additional columnsor rows, respectively, during a single clock cycle. As another example,the embodiments of FIGS. 43 and 44 may include additional row decodersand/or column multiplexers to provide access to additional rows orcolumns, respectively, during a single clock cycle.

Variable Word Length Access in Memory

As used above and further below, the term “coupled” may include directlyconnected, indirectly connected, in electrically communication with, andthe like.

Moreover, terms like “first,” “second,” and the like are used todistinguish between elements or method steps having a same or similarname or title and do not necessarily indicate a spatial or temporalorder.

Typically, a memory chip may include memory banks. The memory banks maybe coupled to a row decoder and a column decoder configured to choose aspecific word (or other fixed size data unit) to be read or written.Each memory bank may include memory cells to store the data units, senseamplifiers to amplify voltages from the memory cells selected by the rowand column decoders, and any other appropriate circuits.

Each memory bank usually has a specific I/O width. For example, the I/Owidth may comprise a word.

While some processes executed by logic circuits using the memory chipmay benefit from using very long words, some other processes may requireonly a part of the word.

Indeed, in-memory computing units (such as processor subunits disposedon the same substrate as the memory chip, e.g., as depicted anddescribed in FIG. 7A) frequently perform memory access operations thatrequire only a part of the word.

To reduce latency associated with accessing an entire word when only aportion is used, embodiments of the present disclosure may provide amethod and a system for fetching only one or more parts of a word,thereby reducing data losses associated with transferring unneeded partsof the word and allowing power saving in a memory device.

Furthermore, embodiments of the present disclosure may also reduce powerconsumption in the interaction between the memory chip and otherentities (such as logic circuits, whether separate like CPUs and GPUs orincluded on the same substrate as the memory chip, such as the processorsubunits depicted and described in FIG. 7A) that access the memory chip,which may receive or write only a part of the word.

A memory access command (e.g., from a logic circuit using the memory)may include an address in the memory. For example, the address mayinclude a row address and a column address or may be translated to a rowaddress and a column address, e.g., by a memory controller of thememory.

In many volatile memories, such as DRAMs, the row address is sent (e.g.,directly by the logic circuit or using the memory controller) to the rowdecoder, which activates the entire row (also called the word line) andloads all of the bitlines included in the row.

The column address identifies the bitline(s) on the activated row thatare transferred outside a memory bank including the bitline(s) and tonext level circuitry. For example, the next level circuitry may comprisean I/O bus of the memory chip. In embodiments using in-memoryprocessing, the next level circuitry may comprise a processor subunit ofthe memory chip (e.g., as depicted in FIG. 7A).

Accordingly, the memory chip described below may be included in orotherwise comprise the memory chip as illustrated in any one of FIG. 3A,3B, 4-6, 7A-7D, 11-13, 16-19, 22, or 23.

The memory chip may be manufactured by a first manufacturing processoptimized for memory cells rather than logic cells. For example, thememory cells manufactured by the first manufacturing process may exhibita critical dimension that is smaller (for example, by a factor thatexceeds 2, 3, 4, 5, 6, 7, 8, 9, 10, and the like), than the criticaldimension of a logic circuit manufactured by the first manufacturingprocess. For example, the first manufacturing process may comprise ananalog manufacturing process, a DRAM manufacturing process, and thelike.

Such a memory chip may comprise an integrated circuit that may include amemory unit. The memory unit may include memory cells, an output port,and read circuitry. In some embodiments, the memory unit may furtherinclude a processing unit, such as a processor subunit as describedabove.

For example, the read circuitry may include a reduction unit and a firstgroup of memory read paths for outputting up to a first number of bitsthrough the output port. The output port may connect to an off-chiplogic circuit (such as an accelerator, CPU, GPU, or the like) or to anon-chip processor subunit, as described above.

In some embodiments, the processing unit may include the reduction unit,may be a part of the reduction unit, may differ from the reduction unit,or may otherwise comprise the reduction unit.

An in-memory read path may be included in the integrated circuit (forexample, may in the memory unit) and may include any circuit and/or linkconfigured for reading from and/or writing to a memory cell. Forexample, the in-memory read path may include a sense amplifier, aconductor coupled to the memory cell, a multiplexer, and the like.

The processing unit may be configured to send to the memory unit a readrequest for reading a second number of bits from the memory unit.Additionally or alternatively, the read request may originate from anoff-chip logic circuit (such as an accelerator, CPU, GPU, or the like).

The reduction unit may be configured to assist in reducing powerconsumption related to an access request, e.g., by using any of thepartial word accesses described herein.

The reduction unit may be configured to control the memory read paths,during a read operation triggered by the read request, based on thefirst number of bits and the second number of bits. For example, thecontrol signal from the reduction unit may affect the memory consumptionof the read paths to reduce energy consumption of memory read paths notrelevant to the requested second number of bits. For example, thereduction unit may be configured to control irrelevant memory read pathswhen the second number is smaller than the first number.

As explained above, the integrated circuit may be included in, mayinclude, or otherwise comprise a memory chip as illustrated in any oneof FIG. 3A, 3B, 4-6, 7A-7D, 11-13, 16-19, 22, or 23.

The irrelevant in-memory read paths may be associated with irrelevantbits of the first number of bits, such as bits of the first number ofbits not included in the second number of bits.

FIG. 50 illustrates an example integrated circuit 5000 including memorycells 5001-5008 of an array 5050 of memory cells, an output port 5020that includes bits 5021-5028, read circuitry 5040 that includes memoryread paths 5011-5018, and reduction unit 5030.

When a second number of bits are read using corresponding memory readpaths, the irrelevant bits of the first number of bits may correspond tobits that should not be read (e.g., bits that are not included in thesecond number of bits).

During the read operation, the reduction unit 5030 may be configured toactivate memory read paths corresponding to the second number of bitssuch that the activated memory read paths may be configured to conveythe second number of bits. In such embodiments, only the memory readpaths corresponding to the second number of bits may be activated.

During the read operation, the reduction unit 5030 may be configured toshut down at least a portion of each irrelevant memory read paths. Forexample, the irrelevant memory read paths may corresponding to theirrelevant bits of the first number of bits.

It should be noted that instead of shutting down at least one portion ofan irrelevant memory path, the reduction unit 5030 may instead insurethat the irrelevant memory path is not activated.

Additionally or alternatively, during the read operation, the reductionunit 5030 may be configured to maintain the irrelevant memory read pathsin a low power mode. For example, a low power mode may comprise a modein which the irrelevant memory paths are supplied with voltage orcurrent lower than a normal operating voltage or current, respectively.

The reduction unit 5030 may be further configured to control bitlines ofthe irrelevant memory read paths.

Accordingly, the reduction unit 5030 may be configured to load bitlinesof relevant memory read paths and maintain bitlines of the irrelevantmemory read paths in the low power mode. For example, only the bitlinesof the relevant memory read paths may be loaded.

Additionally or alternatively, the reduction unit 5030 may be configuredto load bitlines of the relevant memory read paths while maintainingbitlines of the irrelevant memory read paths deactivated.

In some embodiments, the reduction unit 5030 may be configured toutilize portions of the relevant memory read paths during the readoperation and to maintain in the low power mode a portion of eachirrelevant memory read path, wherein the portion differs from a bitline.

As explained above, memory chips may use sense amplifiers to amplifyvoltages from memory cells included therein. Accordingly, the reductionunit 5030 may be configured to utilize portions of the relevant memoryread paths during the read operation and to maintain in the low powermode a sense amplifier associated with at least some of the irrelevantmemory read paths.

In such embodiments, the reduction unit 5030 may be configured toutilize portions of the relevant memory read paths during the readoperation and to maintain in the low power mode one or more senseamplifiers associated with all of the irrelevant memory read paths.

Additionally or alternatively, the reduction unit 5030 may be configuredto utilize portions of the relevant memory read paths during the readoperation and to maintain in the low power mode portions of theirrelevant memory read paths that follow (e.g., spatially and/ortemporally) one or more sense amplifiers associated with the irrelevantmemory read paths.

In any of the embodiments described above, the memory unit may include acolumn multiplexer (not shown).

In such embodiments, the reduction unit 5030 may be coupled between thecolumn multiplexer and the output port.

Additionally or alternatively, the reduction unit 5030 may be embeddedin the column multiplexer.

Additionally or alternatively, the reduction unit 5030 may be coupledbetween the memory cells and the column multiplexer.

The reduction unit 5030 may comprise reduction subunits that may beindependently controllable. For example, different reduction subunitsmay be associated with different memory unit columns.

Although described above with respect to read operations and readcircuitry, and of the embodiments above may similarly be applied forwrite operations and write circuitry.

For example, an integrated circuit according to the present disclosuremay include a memory unit comprising memory cells, an output port, andwrite circuitry. In some embodiments, the memory unit may furtherinclude a processing unit, such as a processor subunit as describedabove. The write circuitry may include a reduction unit and a firstgroup of memory write paths for outputting up to a first number of bitsthrough the output port. The processing unit may be configured to sendto the memory unit a write request for writing a second number of bitsfrom the memory unit. Additionally or alternatively, the write requestmay originate from an off-chip logic circuit (such as an accelerator,CPU, GPU, or the like). The reduction unit 5030 may be configured tocontrol the memory write paths, during a write operation triggered bythe write request, based on the first number of bits and the secondnumber of bits.

FIG. 51 illustrates a memory bank 5100 that include an array 5111 ofmemory cells that are addressed using row and column addresses (e.g.,from an on-chip processor subunit or an off-chip logic circuit, such asan accelerator, CPU, GPU, or the like). As shown in FIG. 51, the memorycells are fed to bitlines (vertical) and word lines (horizontal—manyomitted for simplicity). Moreover, row decoder 5112 may be fed with arow address (e.g., from the on-chip processor subunit, the off-chiplogic circuit, or a memory controller not shown in FIG. 51), columnmultiplexer 5113 may be fed with a column address (e.g., from theon-chip processor subunit, the off-chip logic circuit, or a memorycontroller not shown in FIG. 51), and column multiplexer 5113 mayreceive outputs from up to an entire line and output up to a word overoutput bus 5115. In FIG. 51, the output bus 5115 of the columnmultiplexer 5113 is coupled to a main I/O bus 5114. In otherembodiments, the output bus 5115 may be coupled to a processor subunitof the memory chip (e.g., as depicted in FIG. 7A) sending the row andcolumn addresses. The division of the memory bank into memory mats isnot shown for simplicity.

FIG. 52 illustrates a memory bank 5101. In FIG. 52, the memory bank isalso illustrated as including a PIM (processing in memory) logic 5116that has inputs coupled to output bus 5115. PIM logic 5116 may generateaddresses (e.g., comprising row addresses and column addresses) andoutput the addresses via PIM address buses 5118 to access the memorybank. PIM logic 5116 is an example of a reduction unit (e.g., unit 5030)that also comprises a processing unit. The PIM logic 5016 may controlother circuits not shown in FIG. 52 that assist in the reduction ofpower. PIM logic 5116 may further control the memory paths of a memoryunit including memory bank 5101.

As explained above, the word length (e.g., the number of bitlines chosento be transferred at a time) may be large in some cases.

In those cases, each word for reading and/or writing may be associatedwith a memory path that may consume power at various stages of thereading and/or writing operation, for example:

-   -   a. loading the bitline—to avoid loading the bitline to the        needed value (either from a capacitor on the bitline in a read        cycle or to the new value to be written to the capacitor in a        write cycle), there is a need to disable a sense amplifier        located at the end of the memory array and make sure the        capacitor holding the data is not discharged or charged        (otherwise the data stored thereon would be destructed); and    -   b. moving the data from the sense amplifier through a column        multiplexer that chooses the bitlines and to the rest of the        chip (either to the I/O bus that transfers data in and out of        the chip or to the embedded logic, such as a processor subunit        on the same substrate as the memory, that would use the data).

To achieve power saving, integrated circuits of the present disclosuremay determine, at row activation time, that some parts of a word areirrelevant and then send a disable signal to one or more sense amplifierfor the irrelevant parts of the word.

FIG. 53 illustrates a memory unit 5102 that includes an array 5111 ofmemory cells, a row decoder 5112, a column multiplexer 5113 that iscoupled to output bus 5115, and PIM logic 5116.

Memory unit 5102 also includes switches 5201 that enable or disable thepassage of bits to the column multiplexer 5113. Switches 5201 maycomprise analog switches, transistors configured to function asswitches, or any other circuitry configured to control a supply orvoltage and/or a flow of current to part of memory unit 5102. The senseamplifiers (not shown) may be located at the end of the memory cellarray, e.g., before (spatially and/or temporally) switches 5201.

The switches 5201 may be controlled by enable signals sent over bus 5117from PIM logic 5116. The switches are configured, when disconnected, todisconnect the sense amplifiers (not shown) of the memory unit 5102 andtherefore not discharge or charge bitlines disconnected from the senseamplifiers.

Switches 5201 and PIM logic 5116 may form a reduction unit (e.g.,reduction unit 5030).

In yet another example, PIM logic 5116 may send enable signals to thesense amplifiers (e.g., when the sense amplifiers have an enable input)instead of being sent to switches 5201.

The bitlines may additionally or alternatively be disconnected at otherpoints, e.g., not at the end of the bitlines and after the senseamplifiers. For example—a bitline may be disconnected before enteringthe array 5111.

In these embodiments, power may also be saved on data transfer from thesense amplifiers and forwarding hardware (such as output bus 5115).

Other embodiments (that may save less power but may be easier toimplement) focus on saving the power of the column multiplexer 5113 andtransfer losses from the column multiplexer 5113 to a next levelcircuitry. For example, as explained above, the next level circuitry maycomprise an I/O bus of the memory chip (such as bus 5115). Inembodiments using in-memory processing, the next level circuitry mayadditionally or alternatively comprise a processor subunit of the memorychip (such as PIM logic 5116).

FIG. 54A illustrates a column multiplexer 5113 segmented to segments5202. Each segment 5202 of the column multiplexer 5113 may beindividually enabled or disabled by enable and/or disable signals sentover bus 5119 from PIM logic 5116. Column multiplexer 5113 may also befed by address columns bus 5118.

The embodiment of FIG. 54A may provide better control over differentportions of the output from column multiplexer 5113.

It should be noted that the control of different memory paths may be ofdifferent resolutions, e.g., ranging from a bit resolution and to aresolution of multiple bits. The former may be more effective in senseof power savings. The latter may be simpler to implement and requirefewer control signals.

FIG. 54B illustrates an example method 5130. For example, method 5130may be implemented using any of the memory units described above withrespect to FIG. 50, 51, 52, 53, or 54A.

Method 5130 may include steps 5132 and 5134.

Step 5132 may include sending, by a processing unit (e.g., PIM logic5116) of the integrated circuit and to a memory unit of the integratedcircuit, an access request for reading a second number of bits from thememory unit. The memory unit may include memory cells (e.g., memorycells of array 5111), an output port (e.g., output bus 5115), andread/write circuitry that may include a reduction unit (e.g., reductionunit 5030) and a first group of memory read/write paths for outputtingand/or inputting up to a first number of bits through the output port.

An access request may comprise a read request and/or a write request.

A memory input/output path may comprise a memory read path, a memorywrite path, and/or a path used for both reading and writing.

Step 5134 may include responding to the access request.

For example, step 5134 may include controlling, by the reduction unit(e.g., unit 5030), the memory read/write paths, during an accessoperation triggered by the access request, based on the first number ofbits and the second number of bits.

Step 5134 may further include any one of the following and/or anycombination of any one of the following. Any of the listed belowoperations may be executed during the responding to the access requestbut may also be executed before and/or after responding to the accessrequest.

Thus, step 5134 may include at least one of:

-   -   a. controlling irrelevant memory read paths when the second        number is smaller than the first number, wherein the irrelevant        memory read paths are associated with bits of the first number        of bits not included in the second number of bits;    -   b. activating, during a read operation, relevant memory read        paths, wherein the relevant memory read paths are configured to        convey the second number of bits;    -   c. shutting down, during the read operation, at least a portion        of each one of the irrelevant memory read paths;    -   d. maintaining, during the read operation, the irrelevant memory        read paths in a low power mode;    -   e. controlling bitlines of the irrelevant memory read paths;    -   f. loading bitlines of the relevant memory read paths and        maintaining bitlines of the irrelevant memory read paths in a        low power mode;    -   g. loading bitlines of the relevant memory read paths, while        maintaining bitlines of the irrelevant memory read paths        deactivated;    -   h. utilizing portions of the relevant memory read paths during        the read operation and maintaining in a low power mode a portion        of each irrelevant memory read path, wherein the portion differs        from a bitline;    -   i. utilizing portions of the relevant memory read paths during a        read operation and maintaining in a low power mode a sense        amplifier for at least some of the irrelevant memory read paths;    -   j. utilizing portions of the relevant memory read paths during a        read operation and maintaining in a low power mode a sense        amplifier of at least some of the irrelevant memory read paths;        and    -   k. utilizing portions of the relevant memory read paths during a        read operation and maintaining in a low power mode portions of        the irrelevant memory read paths that follow sense amplifiers of        the irrelevant memory read paths.

A low power mode or an idle mode may comprise a mode in which powerconsumption of a memory access path is lower than power consumption ofthe same when the memory access path is used for an access operation. Insome embodiments, a low power mode may even involve shutting down thememory access path. A low power mode may additionally or alternativelyinclude not activating the memory-access path.

It should be noted that power reductions that occur during the bitlinephase may require that the relevancy or irrelevancy of the memory accesspaths should be known prior to opening the word line. Power reductionsthat occur elsewhere (for example, in the column multiplexer) mayinstead allow for deciding the relevancy or irrelevancy of the memoryaccess paths on every access.

Fast and Low Power Activation and Fast Access Memory

DRAM and other memory types (such as SRAM, Flash, or the like) are oftenbuilt from memory banks, which are usually built to allow for row andcolumn access schemes.

FIG. 55 illustrates an example of a memory chip 5140 that includesmultiple memory mats and associated logic (such as row and columndecoders—depicted as RD and COL in FIG. 55, respectively). In theexample of FIG. 55, the mats are grouped into banks and have word linesand bitlines through them. The memory mats and associated logic aredenoted 5141, 5142, 5143, 5144, 5145 and 5146 in FIG. 55 and share atleast one bus 5147.

Memory chip 5140 may be included in, may include, or otherwise comprisea memory chip as illustrated in any one of FIG. 3A, 3B, 4-6, 7A-7D,11-13, 16-19, 22, or 23.

In DRAM, for example, there is a lot overhead associated with activationof a new row (e.g., preparing a new line for access). Once a line isactivated (also referred to as being opened), the data within that rowmay be available for much faster access. In DRAM, this access may occurin a random manner.

Two problems associated with activating a new line are power and time:

-   -   c. The power rises due to a rush of current caused by accessing        all capacitors on the line together and having to load the line        (e.g., the power can reach several Amperes when opening a line        with just a few memory banks); and    -   d. the time delay problem is mostly associated with the time it        takes to load the row (word) line and then the bit (column)        lines.

Some embodiments of the present disclosure may include a system andmethod to reduce peak power consumption during activation of a line andreduce activation time of the line. Some embodiments may sacrifice fullrandom access within a line, at least to some extent, to reduce thesepower and time costs.

For example, in one embodiment, a memory unit may include a first memorymat, a second memory mat, and an activation unit configured to activatea first group of memory cells included in the first memory mat withoutactivating a second group of memory cells included in the second memorymat. The first group of memory cells and the second group of memorycells may both belong to a single row of the memory unit.

Alternatively, the activation unit may be configured to activate thesecond group of memory cells included in the second memory mat withoutactivating the first group of memory cells.

In some embodiments, the activation unit may be configured to activatethe second group of memory cells after activation of the first group ofmemory cells.

For example, the activation unit may be configured to activate thesecond group of memory cells following expiration of a delay periodinitiated after activation of the first group of memory cells has beencompleted.

Additionally or alternatively, the activation unit may be configured toactivate the second group of memory cells based on a value of a signaldeveloped on a first word line segment coupled to the first group ofmemory cells.

In any of the embodiments described above, the activation unit mayinclude an intermediate circuit disposed between a first word linesegment and a second word line segment. In such embodiments, The firstword line segment may be coupled to the first memory cells and thesecond word line segment may be coupled to the second memory cells.Non-limiting examples of intermediate circuits include switches,flip-flops, buffers, inverters, and the like—some of which areillustrated throughout FIGS. 56-61.

In some embodiments, the second memory cells may be coupled to a secondword line segment. In such embodiments, the second word line segment maybe coupled to a bypass word line path that passes through at least thefirst memory mat. An example of such bypass paths is illustrated in FIG.61.

The activation unit may comprises a control unit configured to control asupply of voltage (and/or a flow of current) to the first group ofmemory cells and to the second group of memory cells based on anactivation signal from a word line associated with the single row.

In another example embodiment, a memory unit may include a first memorymat, a second memory mat, and an activation unit configured to supply anactivation signal to a first group of memory cells of the first memorymat and delay a supply of the activation signal to a second group ofmemory cells of the second memory mat at least until activation of thefirst group of memory cells has been completed. The first group ofmemory cells and the second group of memory cells may belong to a singlerow of the memory unit.

For example, the activation unit may include a delay unit that may beconfigured to delay the supply of the activation signal.

Additionally or alternatively, the activation unit may include acomparator that may be configured to receive the activation signal at aninput of the comparator and to control the delay unit based on at leastone characteristic of the activation signal.

In another example embodiment, a memory unit may include a first memorymat, a second memory mat, and an isolation unit configured to: isolatefirst memory cells of the first memory mat from second memory cells ofthe second memory mat during an initial activation period in which thefirst memory cells are activated; and couple the first memory cells tothe second memory cells following the initial activation period. Thefirst and second memory cells may belong to a single row of the memoryunit.

In the following examples, no modifications to the memory matsthemselves may be required. In certain examples, embodiments may rely onminor modifications to the memory bank.

The diagrams below depict a mechanism to shorten the word signal addedto memory banks, thereby splitting a word line into a number of shorterportions.

In the following figures, various memory bank components were omittedfor clarity.

FIGS. 56-61 illustrate portions (denoted 5140(1), 5140(2), 5140(3),5140(4), 5140(5), and 5149(6), respectively) of memory banks thatinclude row decoder 5112 and multiple memory mats (such as 5150(1),5150(2), 5150(3), 5150(4), 5150(5), 5150(6), 5151(1), 5151(2), 5151(3),5151(4), 5151(5), 5151(6), 5152(1), 5152(2), 5152(3), 5152(4), 5152(5),and 5152(6)) that are grouped within different groups.

Memory mats that are arranged in a row may include different groups.

FIGS. 56-59 and 61 illustrates nine groups of memory mats, where eachgroup includes a pair of memory mats. Any number of groups, each withany number of memory mats, may be used.

Memory mats 5150(1), 5150(2), 5150(3), 5150(4), 5150(5), and 5150(6) arearranged in a row, share multiple memory lines and are divided intothree groups—a first upper group includes memory mats 5150(1) and5150(2), a second upper group includes memory mats 5150(3) and 5150(4),and a third upper group includes memory mats 5150(5) and 5150(6).

Similarly, memory mats 5151(1), 5151(2), 5151(3), 5151(4), 5151(5), and5151(6) are arranged in a row, share multiple memory lines and aredivided into three groups—a first intermediate group includes memorymats 5151(1) and 5151(2), a second intermediate group includes memorymats 5151(3) and 5151(4), and a third intermediate group includes memorymats 5151(5) and 5151(6).

Moreover, memory mats 5152(1), 5152(2), 5152(3), 5152(4), 5152(5) and5152(6) are arranged in a row, share multiple memory lines and aregrouped to three groups—a first lower group includes memory mats 5152(1)and 5152(2), a second lower group includes memory mats 5152(3) and5152(4), and a third lower group includes memory mats 5152(5) and5152(6). Any number of memory mats may be arranged in a row and sharememory lines and may be divided into any number of groups.

For example, the number of memory mats per group may be one, two, or mayexceed two.

As explained above, an activation circuit may be configured to activateone group of memory mats without activating another group of memory matsthat share the same memory lines—or at least are coupled to differentmemory line segments that have a same line address.

FIGS. 56-61 illustrates different examples of activation circuits. Insome embodiments, at least a portion of the activation circuit (such asintermediate circuits) may be located between groups of memory mats toallow memory mats of one group to be activated while another group ofmemory mats of the same row is not activated.

FIG. 56 illustrates intermediate circuits, such as delay or isolationcircuits 5153(1)-5153(3), as positioned between different lines of thefirst upper group of memory and of the second upper group of memorymats.

FIG. 56 also illustrates intermediate circuits, such as delay orisolation circuits 5154(1)-5154(3), as positioned between differentlines of second upper group of memory and of third upper group of memorymats. Additionally, some delay or isolation circuits are positionedbetween groups formed from memory mats of the intermediate groups.Moreover, some delay or isolation circuits are positioned between groupsformed from memory mats of the lower groups.

The delay or isolation circuits may delay or stop a word line signalfrom the row decoder 5112 from propagating along a row to another group.

FIG. 57 illustrates intermediate circuits, such as delay or isolationcircuits, that comprise flip-flops (such as 5155(1)-5155(3) and5156(1)-5156(3)).

When an activation signal is injected to a word line, one of the firstgroups of mats (depending on the word line) is activated while the othergroups along the word line remain deactivated. The other groups may beactivated at the next clock cycle. For example, second groups of theother groups may be activated at the next clock cycle, and third groupsof the other groups may be activated after yet another clock cycle.

The flipflops may comprise D-type flip-flops or any other type offlip-flop. The clock fed to the D-type flip-flop is omitted from thedrawing for simplicity.

Thus, access to the first groups may use power to charge only the partof the word line associated with the first group, which is faster thancharging the entire word line and requires less current.

More than one flip-flop may be used between groups of memory mats,thereby increasing the delay between opening parts. Additionally oralternatively, embodiments may use a slower clock to increase the delay.

Moreover, the groups that are activated may still contain groups fromthe previous line value that was used. For example, the method may allowactivating a new line segment while still accessing data of the previousline, thereby reducing the penalty associated with activating a newline.

Accordingly, some embodiments may have a first group that is activatedand allow other groups of the previously activated line to remain activewith the signals of the bitlines not interfering with each other.

Additionally, some embodiments may include switches and a controlsignals. The control signals may be controlled by the bank controller orby adding flip-flops between control signals (e.g., generating the sametiming effect that the mechanism described above had).

FIG. 58 illustrates intermediate circuits, such as delay or isolationcircuits, that are switches (such as 5157(1)-5157(3) and5158(1)-5158(3)) and positioned between one group of another.

A set of switches positioned between groups may be controlled by adedicated control signal. In FIG. 58 the control signal may be sent by arow control unit 5160(1) and delayed by a sequence of one or more delayunits (e.g., units 5160(2) and 5160(3)) between different sets ofswitches.

FIG. 59 illustrates intermediate circuits, such as delay or isolationcircuits, that are sequences of inverter gates or buffers (such as5159(1)-5159(3) and 5159′1(0-5159′(3)) and positioned between groups ofmemory mats.

Instead of switches, buffers may be used between groups of memory mats.Buffers may allow not dropping voltage along the word line from switchto switch, which is an effect that sometimes occurs when using thesingle transistor structure.

Other embodiments may allow for more random access and still providevery low activation power and time by using added area to the memorybank.

An example is shown in FIG. 60, which illustrates using global wordlines (such as 5152(1)-5152(8)) positioned in proximity to the memorymats. These word lines may or may not pass through the memory mats andare coupled via intermediate circuits, such as switches (such as5157(1)-5157(8)), to word lines within the memory mats. The switches maycontrol which memory mat will be activated and allow a memory controllerto activate, at each point of time, only the relevant line part. Unlikeembodiments using a sequential activation of line portions describedabove, the example of FIG. 60 may provide greater control.

Enable signals, such as row part enable signals 5170(1) and 7150(2), mayoriginate from logic, such as a memory controller, that is not shown.

FIG. 61 illustrates that the global word lines 5180 pass through thememory mats and form bypass paths for the word line signals, which maynot need to be routed outside the mat. Accordingly, the embodimentsshown in FIG. 61 may reduce the area of the memory bank at a cost ofsome memory density.

In FIG. 61, the global world line may pass uninterrupted through amemory mat and may not be connected to memory cells. A local word linesegment may be controlled by one of the switches and connected to memorycells in the mat.

When the groups of memory mats provide a substantial partition of theword lines, the memory bank may virtually support full random access.

Another embodiment for slowing the spreading of the activation signalalong a word line, that also may save some wiring and logic, usesswitches and/or other buffering or isolating circuits between memorymats without using dedicated enable signals and dedicated lines forconveying the enable signals.

For example, a comparator may be used to control switches or otherbuffering or isolating circuits. The comparator may activate the switchor other buffering or isolating circuit when the level of signal on theword line segment monitored by the comparator reaches a certain level.For example, the certain level may indicate that the previous word linesegment was fully loaded.

FIG. 62 illustrates a method 5190 for operating a memory unit. Forexample, method 5130 may be implemented using any of the memory banksdescribed above with respect to FIGS. 56-61.

Method 5190 may include steps 5192 and 5194.

Step 5192 may include activating, by an activation unit, a first groupof memory cells included in a first memory mat of the memory unitwithout activating a second group of memory cells included in a secondmemory mat of the memory unit. The first group of memory cells and thesecond group of memory cells may both belong to a single row of thememory unit.

Step 5194 may include activating, by an activation unit, the secondgroup of memory cells, e.g., after step 5192.

Step 5194 may be executed while the first group of memory cells areactivated, after a full activation of the first group of memory cells,following expiration of a delay period initiated after activation of thefirst group of memory cells has been completed after the first group ofmemory cells are deactivated, and the like.

The delay period may be fixed or may be adjusted. For example, theduration of the delay period may be based on an expected access patternof the memory unit or may be set regardless of the expected accesspattern. The delay period may range between less than one millisecondand more than one second.

In some embodiments, step 5194 may be initiated based on a value of asignal developed on a first word line segment coupled to the first groupof memory cells. For example, when a value of the signal exceeds a firstthreshold, it may indicate that the first group of memory cells arefully activated.

Either one of steps 5192 and 5194 may involve using an intermediatecircuit (e.g., of the activation unit) disposed between a first wordline segment and a second word line segment. The first word line segmentmay be coupled to the first memory cells and the second word linesegment may be coupled to the second memory cells.

Examples of an intermediate circuit are illustrated throughout FIGS.56-61.

Steps 5192 and 5194 may further include controlling, by a control unit,a supply to the first group of memory cells and to the second group ofmemory cells of an activation signal from a word line associated withthe single row.

Using Memory Parallelism to Speedup Testing Times and Testing Logic inMemory Using Vectors

Some embodiments of the present disclosure may speed up testing using inchip testing units.

Generally, memory chips testing requires significant testing time.Reducing testing time can reduce cost of production and also allow formore testing, leading to a more reliable product.

FIGS. 63 and 64 illustrate a tester 5200 and a chip (or a wafer ofchips) 5210. The tester 5200 may include software that manages thetesting. The tester 5200 may run different sequences of data to all ofmemory 5210 and then read the sequences back to identify where failedbits of memory 5210 are located. Once recognized, the tester 5200 mayissue a command to fix the bits, and if it was able to fix the problem,tester 5200 may declare memory 5210 as passed. In other cases, somechips may be declared as failed.

The tester 5200 may write test sequences and then read back the data tocompare it to expected results.

FIG. 64 shows a test system with a tester 5200 and a full wafer 5202 ofchips (such as 5210) being tested in parallel. For example, the tester5200 may connect to each of the chips with a bus of wires.

As shown in FIG. 64, the tester 5200 has to read and write all of thememory chips a few times, and that data must be passed through theexternal chip interface.

Moreover, it may be beneficial to test both logic and memory banks of anintegrated circuit, e.g., using programmable configuration informationthat may be provide using regular I/O operations.

The testing may also benefit from the presence of testing units withinthe integrated circuit.

The testing units may belong to the integrated circuit and may analyze aresults of the test and find, for example, failures in logic (e.g.,processor subunits as depicted in FIG. 7A and described) and/or memory(e.g., across a plurality of memory banks).

Memory testers are usually very simple and exchange test vectors withintegrated circuits according to a simple format. For example, there maybe write vectors that include pairs of addresses of memory entries to bewritten and the values to be written to the memory entries. There mayalso be a read vector that includes addresses of memory entries to beread. At least some of the addresses of the write vectors may be thesame as at least some addresses of the read vectors. At least some otheraddresses of the write vectors may differ from at least some otheraddresses of the read vectors. When programmed, the memory testers mayalso receive an expected result vector that may include the addresses ofmemory entries to be read and the expected values to be read. The memorytester may compare the expected values to the values it reads.

According to an embodiment, the logic (e.g., processor subunits) of anintegrated circuit (with or without the memory of the integratedcircuit) may be tested by a memory tester using the sameprotocol/format. For example, some of the values in the write vector maybe commands to be executed by the logic (and may, for example, involvecalculations and/or memory access) of the integrated circuit. The memorytester may be programmed with the read vector and the expected resultvector that may include memory entry addresses—at least some of whichstore expected values of the calculations. Thus, the memory tester maybe used for testing the logic as well as the memory. Memory testers areusually much simpler and cheaper than logic testers, and the proposedmethods allow for performing complex logic tests using a simple memorytester.

In some embodiments, a logic within the memory may enable testing oflogic within the memory by using only vectors (or other data structures)and not more complex mechanisms common in logic testing (such ascommunicating with the controller, for example, through an interface,telling the logic which circuit to test).

Instead of using testing units, the memory controllers may be configuredto receive instructions to access memory entries included inconfiguration information and execute the access instructions and outputresults.

Any of the integrated circuits illustrated in FIGS. 65-69 may executethe tests—even in the absence of testing units—or in the presence oftesting units not capable to perform tests.

Embodiments of the present disclosure may include a method and systemthat use the parallelism of the memory and the internal chip bandwidthto speed up and improve test times.

The method and system may be based on a memory chip testing itself (asopposed to a tester running the test, reading results of the test, andanalyzing the results), saving the results, and eventually allowing thetester to read them (and, if needed, to program the memory chip back,e.g., to activate redundancy mechanisms). The testing may includetesting the memory or testing the memory banks and the logic (in case ofa computational memory that has functional logic portions to test, suchas that described above in FIG. 7A).

In one embodiments, the method may include reading and writing datawithin the chip such that external bandwidth does not limit the test.

In embodiments where the memory chip includes processor subunits, eachprocessor subunit may be programmed with a test code or configuration.

In embodiments where the memory chip has processor subunits that cannotexecute a test code or is without processor subunits but has memorycontrollers, then the memory controllers may be configured to read andwrite patterns (e.g., programmed to the controllers externally) and marklocations of faults (for example, writing a value to a memory entry,reading the entry, and receiving a value that differs from the writtenvalue) for further analysis.

It should be noted that the testing of a memory may require testing avast number of bits, for example, testing each bit of the memory andverifying that the tested bits are functional. Moreover, sometimes thememory testing may be repeated under different voltage and temperatureconditions.

For some defects, one or more redundancy mechanisms may be activated(e.g., by programming flash or OTP or burning fuses). In addition, thelogic and analog circuits of the memory chips (e.g., controllers,regulators, I/Os) may also have to be tested.

In one embodiment, an integrated circuit that may include a substrate, amemory array disposed on the substrate, a processing array disposed onthe substrate, and an interface disposed on the substrate.

The integrated circuits described herein may be included in, mayinclude, or otherwise comprise a memory chip as illustrated in any oneof FIG. 3A, 3B, 4-6, 7A-7D, 11-13, 16-19, 22, or 23.

FIGS. 65-69 illustrates various integrated circuits 5210 and tester5200.

The integrated circuit is illustrated as including memory banks 5212, achip interface 5211 (such as I/O controller 5214 and bus 5213 shared bythe memory banks), and logic unit (hereinafter “logic”) 5215. FIG. 66illustrates a fuse interface 5216 and a bus 5217 coupled to the fuseinterface and the different memory banks.

FIGS. 65-70 also illustrate various steps in a testing process—such as:

-   -   a. write test sequence 5221 (FIGS. 65, 67, 68 and 69);    -   b. read back test results 5222 (FIGS. 67, 68 and 69);    -   c. write expected results sequence 5223 (FIG. 65);    -   d. read faulty addresses to fix 5224 (FIG. 66); and    -   e. program fuses 5225 (FIG. 66).

Each memory bank may be coupled to and/or controlled by its own logicunit 5215. However, as described above, any allocation of memory banksto logic unit 5215 may be provided. Thus, the number of logic units 5215may differ from the number of memory banks, a logic unit may controlmore than a single memory bank or a fraction of a memory bank, and thelike.

The logic unit 5215 may include one or more testing units. FIG. 65illustrates a testing unit (TU) 5218 within logic 5215. A TU may beincluded in all or some of the logic units 5212. It should be noted thatthe testing unit may be separate from the logic unit or integrated withthe logic unit.

FIG. 65 also illustrates a test patter generator (denoted GEN) 5219within TU 5218.

A test pattern generator may be included in all or some of the testingunits. For simplicity, test patter generators and testing units are notillustrated in FIGS. 66-70 but may be included in such embodiments.

The memory array may include multiple memory banks. Moreover, theprocessing array may include a plurality of testing units. The pluralityof testing units may be configured to test the multiple memory banks toprovide test results. The interface may be configured to output, to adevice external to the integrated circuit, information indicative of thetest results.

The plurality of testing units may include at least one test patterngenerator configured to generate at least one test pattern for use intesting one or more of the multiple memory banks. In some embodiments,as explained above, each of the plurality of testing units may include atest pattern generator configured to generate a test pattern for use bya particular one of the plurality of testing units to test at least oneof the multiple memory banks. As indicated above, FIG. 65 illustrates atest pattern generator (GEN) 5219 within a testing unit. One or more oreven all logic units may include the test pattern generator.

The at least one test pattern generator may be configured to receiveinstructions from the interface for generating the at least one testpattern. A test pattern may include memory entries that should beaccessed (e.g., read and/or written) during a test and/or values to bewritten to the entries, and the like.

The interface may be configured to receive, from an external unit thatmay be external to the integrated circuit, configuration informationincluding the instructions for generating the at least one test pattern.

The at least one test pattern generator may be configured to readconfiguration information including instructions for generating the atleast one test pattern from the memory array.

In some embodiments, the configuration information may include a vector.

The interface may be configured to receive, from a device that may beexternal to the integrated circuit, configuration information that mayinclude instructions that may be the at least one test pattern.

For example, at least one test pattern may include memory array entriesto be accessed during the testing of the memory array.

The at least one test pattern further may include input data to bewritten to the memory arrays entries accessed during the testing of thememory array.

Additionally or alternatively, at least one test pattern further mayinclude input data to be written to the memory array entries accessedduring the testing of the memory array and expected values of outputdata to be read from the memory array entries accessed during thetesting of the memory array.

In some embodiments, the plurality of testing units may be configured toretrieve, from the memory array, test instructions that once executed bythe plurality of testing units cause the plurality of testing units totest the memory array.

For example, the test instructions may be included in configurationinformation.

The configuration information may include expected results of thetesting of the memory array.

Additionally or alternatively, the configuration information may includevalues of output data to be read from memory array entries accessedduring the testing of the memory array.

Additionally or alternatively, the configuration information may includea vector.

In some embodiments, the plurality of testing units may be configured toretrieve, from the memory array, test instructions that once executed bythe plurality of testing units cause the plurality of testing units totest the memory array and to test the processing array.

For example, the test instructions may be included in configurationinformation.

The configuration information may include a vector.

Additionally or alternatively, the configuration information may includeexpected results of the testing of the memory array and of theprocessing array.

In some embodiments, as described above, the plurality of testing unitsmay lack a test pattern generator for generating a test pattern usedduring the testing of the multiple memory banks.

In such embodiments, at least two of the plurality of testing units maybe configured to test in parallel at least two of the multiple memorybanks.

Alternatively, at least two of the plurality of testing units may beconfigured to test in series at least two of the multiple memory banks.

In some embodiments, the information indicative of the test results mayinclude identifiers of faulty memory array entries.

In some embodiments, the interface may be configured to retrievemultiple times, during the testing of the memory array, partial testresults obtained by the plurality of testing circuits.

In some embodiments, the integrated circuit may include an errorcorrection unit configured to correct at least one error detected duringthe testing of the memory array. For example, the error correction unitmay be configured to fix memory errors using any appropriate technique,for example, by disabling some memory words and replacing them withredundant words.

In any of the embodiments described above, the integrated circuit may bea memory chip.

For example, integrated circuit may include a distributed processor,wherein the processing array may include a plurality of subunits of thedistributed processor, as depicted in FIG. 7A.

In such embodiments, each one of the processor subunits may beassociated with a corresponding, dedicated one of multiple memory banks.

In any of the embodiments described above, the information indicative ofthe test results may indicate a status of at least one memory bank. Thestatus of a memory bank may be provided in one or more granularities—permemory word, per a group of entries, or per the entire memory bank.

FIGS. 65-66 illustrates four steps in a tester testing phase.

In the first step, the tester writes (5221) the test sequence and thelogic units of the banks write the data to their memories. The logic mayalso be complex enough to receive a command from the tester and generatethe sequence on its own (as explained below).

In the second step, the tester writes (5223) to the tested memory theexpected results and the logic units compare the expected results todata read from their memory banks, saving a list of errors. Writing theexpected results may be simplified if the logic is complex enough togenerate on its own the sequence of expected results (as explainedbelow).

In the third step, the tester reads (5224) from the logic units thefaulty addresses.

In the fourth step, the tester acts (5225) upon the results and can fixthe errors. For example, it may connect to a specific interface toprogram fuses in the memory but can also use any other mechanism thatallows for programming an error correction mechanism within the memory.

In such embodiments, the memory testers may use vectors to test thememory.

For example, each vector may be built from an input series and an outputseries.

The input series may include pairs of address and data to write to thememory (in many embodiments, this series could be modeled as a formulathat allows a program, such as one executed by the logic units, togenerate it when needed).

In some embodiments, a test pattern generator may generate such vectors.

It should be noted that a vector is an example data structure but someembodiments may use other data structures. The data structures may becompliant with other test data structures generated by testers locatedoutside the integrated circuit.

The output series may include address and data pairs comprising expecteddata to be read back from the memory (in some embodiments, the seriescould additionally or alternatively be generated by a program atruntime, e.g., by the logic units).

Memory testing usually includes executing a list of vectors, each vectorwriting data to the memory according to the input series and thenreading data back according to the output series and comparing it to itsexpected data.

In case of a mismatch, the memory may be either classifies as faulty or,if the memory includes mechanisms for redundancy, may have theredundancy mechanisms activated such that the vectors are tested againon the activated redundancy mechanisms.

In embodiments where memories include processor subunits (as describedabove with respect to FIG. 7A) or contain many memory controllers, theentire test may be handled by the logic units of the banks Thus, amemory controller or processor subunit may perform the tests.

The memory controller may be programmed from the tester, and the resultsof the test may be saved in the controller itself to later be read bythe tester.

To configure and test the operation of the logic unit, the tester mayconfigure the logic unit for memory access and confirm that the resultscan be read by the memory access.

For example, an input vector may contain programming sequences for thelogic unit, and the output vector may contain expected results of suchtesting. For example, if a logic unit such as a processor subunitcomprises a multiplier or adder configured to perform computations ontwo addresses in the memory, an input vector may include a set ofcommands that writes data to the memory and a set of commands to theadder/multiplier logic. As long as the adder/multiplier results can beread back to an output vector, the results may be sent to the teste.

The testing may further include loading the logic configuration from thememory and having the logic output sent to the memory.

In embodiments where the logic unit loads its configuration from thememory (e.g., if the logic is a memory controller), the logic unit mayrun its code from the memory itself.

Accordingly, the input vector may include a program for the logic unit,and the program itself may test various circuits in the logic unit.

Thus, the testing may not be limited to receiving vectors in formatsused by external testers.

If the commands that are loaded to the logic unit instruct the logicunit to write back results into the memory bank, then the tester mayread those results and compare them to an expected output series.

For example, the vector written to the memory may be or may include atest program for the logic unit (e.g., the testing may assume the memoryis valid, but even if not, the test program written would not work, andthe test would fail, which is an acceptable result since the chip isinvalid anyway) and/or how the logic unit ran the code and wrote backthe results to the memory. Since all testing of the logic unit may bedone through the memory (e.g., writing logic test inputs to the memoryand writing test results back to the memory), the tester may run asimple vector test with an input sequence and expected output sequence.

Logic configuration and results may be accessed as read and/or writecommands.

FIG. 68 illustrates a tester 5200 that sends a write test sequence 5221that is a vector.

Parts of the vector include test code 5232 that is split between memorybanks 5212 that are coupled to logic 5215 of a processing array.

Each logic 5215 may execute the code 5232 stored in its associatedmemory bank, and the execution may include accessing one or more memorybanks, performing calculations, and storing the results (e.g., testresults 5231) in the memory banks 5212.

The test results may be sent back (e.g., read back results 5222) bytester 5200.

This may allow logic 5215 to be controlled by commands received by theI/O controller 5214.

In FIG. 68, the I/O controller 5214 is connected to the memory banks andto the logic. In other embodiments, logic may be connected between theI/O controller 5214 and the memory banks.

FIG. 70 illustrates a method 5300 for testing memory banks. For example,method 5300 may be implemented using any of the memory banks describedabove with respect to FIGS. 65-69.

Method 5300 may include steps 5302, 5310, and 5320. Step 5302 mayinclude receiving a request to test memory banks of an integratedcircuit. The integrated circuit may include a substrate, a memory arraythat is disposed on the substrate and comprises the memory banks, aprocessing array disposed on the substrate, and an interface disposed onthe substrate. The processing array may include a plurality of testingunits, as described above.

In some embodiments, the request may include configuration information,one or more vectors, commands, and the like.

In such embodiments, the configuration information may include expectedresults of the testing of the memory array, instructions, data, valuesof output data to be read from memory array entries accessed during thetesting of the memory array, a test pattern, and the like.

The test pattern may include at least one out of (i) memory arrayentries to be accessed during the testing of the memory array, (ii)input data to be written to the memory arrays entries accessed duringthe testing of the memory array, or (iii) expected values of output datato be read from the memory array entries accessed during the testing ofthe memory array.

Step 5302 may include at least one of the following and/or may followedby at least one of the following:

-   -   a. receiving by the at least one test pattern generator        instructions from the interface for generating the at least one        test pattern;    -   b. receiving by the interface and from an external unit that is        external to the integrated circuit, configuration information        including the instructions for generating the at least one test        pattern;    -   c. reading, by the at least one test pattern generator,        configuration information including instructions for generating        the at least one test pattern from the memory array;    -   d. receiving, by the interface and from an external unit that is        external to the integrated circuit, configuration information        that comprises instructions that are the at least one test        pattern;    -   e. retrieving, by a plurality of testing units and from the        memory array, test instructions that, once executed by the        plurality of testing units, cause the plurality of testing units        to test the memory array; and    -   f. receiving by the plurality of testing units and from the        memory array, test instructions that, once executed by the        plurality of testing units, cause the plurality of testing units        to test the memory array and to test the processing array.

Step 5302 may be followed by step 5310. Step 5310 may include testing,by the plurality of testing units and in response to the request, themultiple memory banks to provide test results.

Method 5300 may further include receiving, by the interface, a pluralityof times, during the testing of the memory array, partial test resultsobtained by the plurality of testing circuits.

Step 5310 may include at least one of the following and/or may followedby at least one of the following:

-   -   a. generating, by one or more test pattern generators (e.g.,        included in one, some, or all of the plurality of testing units)        test patterns for use by one or more testing units to test at        least one of the multiple memory banks;    -   b. testing in parallel, by at least two of the plurality of        testing units, at least two of the multiple memory banks;    -   c. testing in series, by at least two of the plurality of        testing units, at least two of the multiple memory banks;    -   d. writing values to memory entries, reading the memory entries,        and comparing the results; and    -   e. correcting, by an error correction unit, at least one error        detected during the testing of the memory array.

Step 5310 may be followed by step 5320. Step 5320 may includeoutputting, by the interface and outside the integrated circuit,information indicative of the test results.

The information indicative of the test results may include identifiersof faulty memory array entries. This may save time by not sending readdata regarding each memory entry.

Additionally or alternatively, the information indicative of the testresults may indicate a status of at least one memory bank.

Accordingly, in some embodiments, the information indicative of the testresults may be much smaller than the aggregate size of data unitswritten to the memory banks of read from the memory banks during thetesting and may be much smaller than the input data that may be sentfrom a tester that tests the memory without an assistance of the testingunit.

The tested integrated circuit may comprise memory chip and/or adistributed processor as illustrated in any of the previous figures. Forexample, the integrated circuits described herein may be included in,may include, or otherwise comprise a memory chip as illustrated in anyone of FIG. 3A, 3B, 4-6, 7A-7D, 11-13, 16-19, 22, or 23.

FIG. 71 illustrates an example of method 5350 for testing memory banksof an integrated circuit. For example, method 5350 may be implementedusing any of the memory banks described above with respect to FIGS.65-69.

Method 5350 may include steps 5352, 5355, and 5358. Step 5352 mayinclude receiving by an interface of an integrated circuit,configuration information that comprises instructions. The integratedcircuit that includes the interface may also include a substrate, amemory array that comprises memory bank and is disposed on thesubstrate, a processing array disposed on the substrate; and aninterface disposed on the substrate.

The configuration information may include expected results of thetesting of the memory array, instructions, data, values of output datato be read from memory array entries accessed during the testing of thememory array, a test pattern, and the like.

Additionally or alternatively, the configuration information may includethe instructions, addresses of memory entries to write the instructions,input data, and may also include addresses of memory entries to receiveoutput values calculated during the execution of the instructions.

The test pattern may include at least one out of (i) memory arrayentries to be accessed during the testing of the memory array, (ii)input data to be written to the memory arrays entries accessed duringthe testing of the memory array, or (iii) expected values of output datato be read from the memory array entries accessed during the testing ofthe memory array.

Step 5352 may be followed by step 5355. Step 5355 may include executing,by the processing array, the instructions by accessing the memory array,performing computational operations, and providing results.

Step 5355 may be followed by step 5358. Step 5358 may includeoutputting, by the interface and outside the integrated circuit,information indicative of the results.

The foregoing description has been presented for purposes ofillustration. It is not exhaustive and is not limited to the preciseforms or embodiments disclosed. Modifications and adaptations will beapparent to those skilled in the art from consideration of thespecification and practice of the disclosed embodiments. Additionally,although aspects of the disclosed embodiments are described as beingstored in memory, one skilled in the art will appreciate that theseaspects can also be stored on other types of computer readable media,such as secondary storage devices, for example, hard disks or CD ROM, orother forms of RAM or ROM, USB media, DVD, Blu-ray, 4K Ultra HD Blu-ray,or other optical drive media.

Computer programs based on the written description and disclosed methodsare within the skill of an experienced developer. The various programsor program modules can be created using any of the techniques known toone skilled in the art or can be designed in connection with existingsoftware. For example, program sections or program modules can bedesigned in or by means of .Net Framework, .Net Compact Framework (andrelated languages, such as Visual Basic, C, etc.), Java, C++,Objective-C, HTML, HTML/AJAX combinations, XML, or HTML with includedJava applets.

Moreover, while illustrative embodiments have been described herein, thescope of any and all embodiments having equivalent elements,modifications, omissions, combinations (e.g., of aspects across variousembodiments), adaptations and/or alterations as would be appreciated bythose skilled in the art based on the present disclosure. Thelimitations in the claims are to be interpreted broadly based on thelanguage employed in the claims and not limited to examples described inthe present specification or during the prosecution of the application.The examples are to be construed as non-exclusive. Furthermore, thesteps of the disclosed methods may be modified in any manner, includingby reordering steps and/or inserting or deleting steps. It is intended,therefore, that the specification and examples be considered asillustrative only, with a true scope and spirit being indicated by thefollowing claims and their full scope of equivalents.

1. A memory chip comprising: a plurality of memory banks; a data storageconfigured to store access information indicative of access operationsfor one or more segments of the plurality of memory banks; and a refreshcontroller configured to perform a refresh operation of the one or moresegments based, at least in part, on the stored access information. 2.The memory chip of claim 1, wherein the data storage includes atimestamp register configured to store a refresh time indicatorindicative of a time at which the one or more segments of the pluralityof memory banks were last refreshed.
 3. The memory chip of claim 2,wherein the refresh controller is configured to skip a refresh operationrelative to the one or more segments of the plurality of memory banks ifthe refresh time indicator indicates a last refresh time within apredetermined time threshold.
 4. The memory chip of claim 3, wherein therefresh controller, after skipping a refresh operation relative to theone or more segments, is configured to alter the stored refresh timeindicator associated with the one or more segments such that during anext operation cycle, the one or more segments will be refreshed.
 5. Thememory chip of claim 1, wherein the one or more segments include atleast one row of memory structures within the plurality of memory banks.6. The memory chip of claim 1, further comprising a timer, and whereinthe refresh controller is configured to use an output of the timer inclearing the access information stored in the data storage after athreshold time interval.
 7. The memory chip of claim 1, wherein the datastorage comprises at least one bit register associated with the one ormore segments, wherein a state of the at least one bit register isconfigured to be changed in response to the one or more segments beingaccessed, and wherein the refresh controller is configured to skip arefresh of the one or more segments based on the state of the at leastone bit register.
 8. The memory chip of claim 7, wherein access to theone or more segments includes a write operation associated with the oneor more segments.
 9. The memory chip of claim 7, further comprising atimer, and wherein the refresh controller is configured to reset the atleast one bit register after a threshold time interval.
 10. The memorychip of claim 7, wherein the refresh controller comprises a row counterand an adder configured to assist in updating a bit table associatedwith the plurality of memory banks based, at least in part, on the stateof the at least one bit register, and wherein the refresh controllerfurther includes a refresh gate configured to control whether a refreshto the one or more segments occurs based a corresponding value stored inthe bit table.
 11. The memory chip of claim 7, further comprising atimer, and wherein the refresh controller is configured to use an outputof the timer in clearing the access information stored in the datastorage after a threshold time interval.
 12. The memory chip of claim 1,wherein the plurality of memory banks comprise dynamic random accessmemory (DRAM).
 13. The memory chip of claim 7, further comprising asense amplifier configured to access the one or more segments andcommunicate with the refresh controller configured to update the stateof the at least one bit register.
 14. The memory chip of claim 7,wherein the refresh controller is integrated with a memory controllerfor the plurality of memory banks.
 15. The memory chip of claim 1,further comprising: a processing array disposed on a same substrate asthe plurality of memory banks, the processing array including aplurality of processor subunits, each one of the processor subunitsbeing associated with a corresponding, dedicated one of the plurality ofmemory banks; a first plurality of buses, each connecting one of theplurality of processor subunits to its corresponding, dedicated memorybank; and a second plurality of buses, each connecting one of theplurality of processor subunits to another of the plurality of processorsubunits.
 16. The memory chip of claim 1, further comprising: aprocessing array disposed on a same substrate as the plurality of memorybanks, the processing array including a plurality of logic portions eachincluding an address generator, wherein each one of the addressgenerators is associated with a corresponding, dedicated one of theplurality of memory banks; and a plurality of buses, each connecting oneof the plurality of address generators to its corresponding, dedicatedmemory bank.
 17. The memory chip of claim 1, further comprising: aprocessing array disposed on a same substrate as the plurality of memorybanks, the processing array including a plurality of processor subunits,each one of the processor subunits being associated with acorresponding, dedicated one of the plurality of memory banks; and aplurality of buses, each one of the plurality of buses connecting one ofthe plurality of processor subunits to at least another one of theplurality of processor subunits, wherein the plurality of buses are freeof timing hardware logic components such that data transfers betweenprocessor subunits and across corresponding ones of the plurality ofbuses are uncontrolled by timing hardware logic components.
 18. Anon-transitory computer-readable medium storing instructions that, whenexecuted by at least one processor, cause the at least one processor to:receive higher-level computer code; identify a plurality of memorysegments distributed over a plurality of memory banks associated with amemory chip to be accessed by the higher-level computer code; assess thehigher-level computer code to identify a plurality of memory readcommands to occur over a plurality of memory access cycles; and cause adistribution of data, associated with the plurality of memory readcommands, across each of the plurality of memory segments such that eachof the plurality of memory segments is accessed during each of theplurality of memory access cycles.
 19. A non-transitorycomputer-readable medium storing instructions that, when executed by atleast one processor, cause the at least one processor to: receivehigher-level computer code; identify a plurality of memory segmentsdistributed over a plurality of memory banks associated with a memorychip to be accessed by the higher-level computer code; assess thehigher-level computer code to identify a plurality of memory accesscommands each implicating one or more of the plurality of memorysegments; based on analysis of the memory access commands and for eachmemory segment among the plurality of memory segments, track an amountof time that would accrue from a last access to the memory segment; andin response to a determination that an amount of time since a lastaccess for any particular memory segment would exceed a predeterminedthreshold, introduce into the higher-level computer code at least one ofa memory refresh command or a memory access command configured to causean access to the particular memory segment.
 20. A memory chip having asoftware-configurable memory refresh control, the memory chipcomprising: a plurality of memory banks and a plurality of memorysegments included in each of the plurality of memory banks; and arefresh controller including a data storage element storing at least onememory refresh pattern to be implemented in refreshing the plurality ofmemory segments included in each of the plurality of memory banks, andwherein the memory refresh pattern is configurable using software toidentify which of the plurality of memory segments in a particularmemory bank are to be refreshed during a refresh cycle and which of theplurality of memory segments in the particular memory bank are not to berefreshed during the refresh cycle.
 21. The memory chip of claim 20,wherein the refresh pattern comprises a table including a plurality ofmemory segment identifiers, and wherein the plurality of memory segmentidentifiers are assigned by the software to identify ranges of theplurality of memory segments in a particular memory bank that are to berefreshed during a refresh cycle and ranges of the plurality of memorysegments in the particular memory bank that are not to be refreshedduring the refresh cycle.
 22. The memory chip of claim 21, wherein theplurality of memory segments are arranged in rows, and each memorysegment identifier is configured to identify a particular locationwithin a row of memory where a memory refresh should either begin orend.
 23. The memory chip of claim 21, wherein the data storage elementof the refresh controller includes a plurality of memory refreshpatterns each representing a different refresh pattern for refreshingthe plurality of memory segments included in each of the plurality ofmemory banks, and wherein the refresh controller is configured to allowselection of which of the plurality of memory refresh patterns toimplement during a particular refresh cycle.
 24. The memory chip ofclaim 21, wherein the data storage element comprises a plurality ofconfigurable registers.
 25. A method for refreshing memory banks on amemory chip, the method comprising: accessing a data storage containinginformation indicative of access operations for one or more segments ofa plurality of memory banks included on the memory chip; and performinga refresh operation, with a refresh controller, of the one or moresegments based, at least in part, on the stored information indicativeof access operations.
 26. The method of claim 25, further includingskipping a refresh operation relative to the one or more segments of theplurality of memory banks if a refresh time indicator indicates a lastrefresh time within a predetermined time threshold, wherein the refreshtime indicator is indicative of a time at which the one or more segmentsof the plurality of memory banks were last refreshed, and wherein therefresh time indicator is stored in a timestamp register of the datastorage.
 27. The method of claim 26, further including: causing therefresh controller, after skipping a refresh operation relative to theone or more segments, to alter the stored refresh time indicatorassociated with the one or more segments such that during a nextoperation cycle, the one or more segments will be refreshed.
 28. Themethod of claim 25, further including causing the refresh controller touse an output of a timer associated with the memory chip in clearing theaccess information stored in the data storage after a threshold timeinterval.
 29. The method of claim 25, wherein the data storage comprisesat least one bit register associated with the one or more segments, andwherein the method includes changing a state of the at least one bitregister in response to the one or more segments being accessed, andcausing the refresh controller to skip a refresh of the one or moresegments based on the state of the at least one bit register.
 30. Themethod of claim 29, further including causing the refresh controller toreset the at least one bit register after a threshold time intervalbased on an output of a timer associated with the memory chip. 31-225.(canceled)